CanJPsychiatry 2015;60(2 Suppl 1):S55–S60
Chapter 6
Parent–Youth Agreement on Self-Reported Competencies of Youth With Depressive and Suicidal Symptoms Valentin Mbekou, PhD1; Sasha MacNeil, BA2; Martin Gignac, MD, FRCPC3; Johanne Renaud, MD, MSc, FRCPC4 1
Psychologist, Standard Life Centre for Breakthroughs in Teen Depression and Suicide Prevention, Douglas Mental Health University Institute, Montréal, Québec.
2
Student, Psychology Department, Concordia University, Montréal, Québec; Research Assistant, Standard Life Centre for Breakthroughs in Teen Depression and Suicide Prevention, Douglas Mental Health University Institute, Montréal, Québec.
3
Child and Adolescent Psychiatrist, Philippe Pinel Institute, Montréal, Québec.
4
Child and Adolescent Psychiatrist, Standard Life Centre for Breakthroughs in Teen Depression and Suicide Prevention, Douglas Mental Health University Institute, Montréal, Québec; Medical Chief, McGill Group for Suicide Studies, McGill University, Montréal, Québec. Correspondence: Douglas Mental Health University Institute, 6875 Lasalle Boulevard, Montréal, QC H4H 1R3; [email protected].
Key Words: adolescence, depression, psychopathology, inter-informant agreement, risk factors, protective factors, competencies Received January 2014, revised, and accepted March 2014.
Objective: A multi-informant approach is often used in child psychiatry. The Achenbach System of Empirically Based Assessment uses this approach, gathering parent reports on the Child Behaviour Checklist (CBCL) and youth reports on the Youth Self-Report (YSR), which contain scales assessing both the child’s problems and competencies. Agreement between parent and youth perceptions of their competencies on these forms has not been studied to date. Method: Our study examined the parent–youth agreement of competencies on the CBCL and YSR from a sample of 258 parent–youth dyads referred to a specialized outpatient clinic for depressive and suicidal disorders. Intraclass correlation coefficients were calculated for all competency scales (activity, social, and academic), with further examinations based on youth’s sex, age, and type of problem. Results: Weak-to-moderate parent–youth agreements were reported on the activities and social subscales. For the activities subscale, boys’ ratings had a strong correlation with parents’ ratings, while it was weak for girls. Also, agreement on activities and social subscales was stronger for dyads with the youth presenting externalizing instead of internalizing problems.
open access
Conclusion: Agreement on competencies between parents and adolescents varied based on competency and adolescent sex, age, and type of problem. WWW
Accord parents–adolescents sur les compétences auto-déclarées d’adolescents présentant des symptômes dépressifs et suicidaires Objectif : Une approche multi-informant est souvent utilisée en pédopsychiatrie. Le Système d’évaluation empirique Achenbach utilise cette approche qui recueille les déclarations des parents à la Child Behaviour Checklist (CBCL) et celles des adolescents au Youth Self-Report (YSR), lesquels contiennent des échelles qui évaluent les problèmes et les compétences de l’enfant. L’accord entre les perceptions des parents et des adolescents relativement à leurs compétences à ces mesures n’a pas été étudié jusqu’ici. Méthode : Notre étude a examiné l’accord parents–adolescents sur les compétences à la CBCL et à l’YSR dans un échantillon de 258 dyades parents–adolescents adressé à une clinique ambulatoire spécialisée dans les troubles dépressifs et suicidaires. Les coefficients de corrélation intraclasse ont été calculés pour toutes les échelles de compétence (activité, sociale, et scolaire), et d’autres examens selon le sexe, l’âge, et le type de problème de l’adolescent. Résultats : Des accords parents–adolescents faibles à modérés ont été observés aux souséchelles d’activités et sociale. Pour la sous-échelle d’activités, les cotes des garçons étaient fortement corrélées à celles des parents, tandis que les corrélations étaient faibles pour les filles. Aussi, aux sous-échelles d’activités et sociale, l’accord était plus prononcé dans les dyades où l’adolescent présentait des problèmes d’externalisation plutôt que d’internalisation. Conclusion : L’accord sur les compétences entre les parents et les adolescents variait selon la compétence et le sexe, l’âge, et le type de problème.
www.TheCJP.ca
The Canadian Journal of Psychiatry, Vol 60, Supplement 1, February 2015 W S55
Chapter 6
I
n child psychiatry, it is common practice to consult multiple informants’ ratings during assessment,1 thereby gathering information from the youth, their parents, and, in some cases, their teachers. Each informant additional to the child is believed to contribute a new perspective on the child’s behaviour depending on the amount of time spent with the child, the individual’s closeness to the child, the situation of the observation, and the purpose of the assessment.2 The study of cross-informant agreement has been of interest to provide effective interventions based on a more complete conceptualization of the youth’s functioning.3
An important tool in child psychiatry is the ASEBA,4 which contains various questionnaires using a multi-informant approach to assess the youth’s functioning. Among the questionnaires comprising the ASEBA, youths provide their own perception of their personal experience on the YSR form while parents complete a comparable form, the CBCL, regarding their perceptions of their children. Research evaluating agreement across multiple informants has repeatedly shown differences between self-reported and parent-reported behaviours and problems in youths. Among clinically referred samples, it has been found that parents report as many or more externalizing problems than do the youths, likely due to the easily observable and public nature of associated behaviours.1,2,5 It has been suggested that adolescents are not the most reliable source for indicating externalizing problems,6 possibly because of a tendency to minimize the importance of their problems on behaviourrating checklists, or because of denial or lack of insight.7 Conversely, for internalizing problems parental reports tend to underestimate the severity, seeing as youth will report more internalizing problems.2,5 Internalizing problems, therefore, seem to be poorly recognized by parents due to their less observable and more private nature,1,8 and the youths are a better source of this kind of information. Cross-informant agreement is usually expected to be greater for externalizing problems than internalizing problems for this reason, yet mixed results have been found.5,8 Opposite patterns of crossinformant agreement have been observed among nonclinical youth samples; however, youths tend to self-report higher severity ratings and more problems of both kinds in comparison to their parents.1,5,8,9 These findings have been replicated across cultures,5 yet further examination within these patterns of agreement has shown differences among them based on youth characteristics, such as age and sex. Edelbrock et al10 found differences in parent–child agreement on a structured interview that were contingent with the age of the child, such that agreement was stronger among dyads with older youths (14 to 18 years) than those with younger youths (10 to 13 years). These results were also found using the ASEBA’s CBCL and YSR.11 Abbreviations ASEBA Achenbach System of Empirically Based Assessment CBCL
Child Behaviour Checklist
ICC
intraclass correlation coefficient
YSR
Youth Self-Report
S56 W La Revue canadienne de psychiatrie, vol 60, supplément 1, février 2015
Clinical Implications •
Clinicians should consider risk and protective factors in the shaping of interventions.
•
Parental reports of youth competencies can be used with weak-to-moderate reliability.
Limitations •
Our study is limited by the Berkson bias.
•
Competencies reported by parents on the CBCL may not reflect the youths’ actual competencies.
Most research and clinical work to date in the field of adolescent mental health, including the use of the CBCL and YSR, has focused on the identification and treatment of various forms of problems and psychopathologies.6 More recently, there has been growing interest for the role of protective factors and well-being in mental health outcomes and their integration into etiological models of mental health to provide a more complete understanding of functioning.12 Studies6,12,13 have shown the relevance of including well-being as a dimension in mental health models. The ASEBA tools contain both these dimensions, whereby youths are scored on both problem and competencies subscales. The CBCL and YSR contain 2 super-ordinate problem subscales to assess internalizing and externalizing problems, while further assessing 3 competency subscales (activities, social competence, and academics) to obtain an overarching total competence score.14 The activities subscale explores the youth’s implications in sports and nonsports‒related activities (how many and how implicated they are within these activities), as well as jobs and job performance. The social competence subscale examines involvement within groups and organizations; involvement with friend groups; behaviour with peers, siblings and family; and amount of time spent alone. The academic subscale evaluates school performance (failures, class rank), whether the youth has repeated grades, is in remedial class, or if other school problems are reported.14 Research has shown that the competencies explored by the CBCL and YSR are related to positive outcomes in adolescent psychopathology. More involvement in activities (sports, social, and academic) during high school has been associated with lower levels of externalizing symptoms, along with less substance use and behaviour disorders.15 Difficulties with peers and (or) teachers, preparing for examinations, reading and writing tasks, self-reliant school performance, and other school performance issues have been associated with elevated depression among both boys and girls.16 Effective social competence has been linked with reduced likelihood of internalizing problems.17 Therefore, the assessment of such competencies may provide information about outcomes beyond the present risk factors. The ASEBA assessment tools competency scales have not been studied as extensively as the problem scales. The few studies examining the competency scales have reported mixed results. Study of the competencies has found sex differences on the social competence and total competence www.LaRCP.ca
Parent–Youth Agreement on Self-Reported Competencies of Youth With Depressive and Suicidal Symptoms
scales18,19 and others have not.20,21 Age effects were also reported by some, whereby adolescents aged 17–18 years were found to be rated higher on the activity subscale and on total competence than younger adolescents,18 while an inverse relation has been reported by others.21 A few studies have examined cross-informant agreement (including reports from self, peers, parents, and teachers) of youth competencies.22‒24 A meta-analysis conducted by Renk and Phares3 reported a small correlation (r = 0.21) between self-reports and parental reports of social competence, with greater agreement among dyads with younger children than with older ones. These studies of cross-informant agreement have not, however, used the CBCL or YSR from the ASEBA. In light of the importance of well-being on mental health outcomes and the widespread use of the CBCL and YSR in child psychiatry, our study aimed to examine the agreement between parents and youths for youth competencies as reported on the CBCL and YSR. Only a few studies have examined this. Further investigation into youth characteristics that may affect agreement was also undertaken to evaluate whether the degree of agreement would change depending on the youth’s age, sex, and the types of problems they present.
Method Participants
All youths who were included in the study were referred from first- or second-line health clinics to the Depressive and Suicidal Disorders Outpatient Clinic (youth section) at the Douglas Mental Health University Institute because of depressive symptoms. Permission to access medical files of patients followed from 2006 to 2013 was granted through the Directorate of Professional and Hospital Services of the Douglas Mental Health University Institute. A total of 258 parent‒child dyads were included in our study. Upon entering the specialized outpatient clinic, the CBCL and YSR were administered to the parent(s) and the youth as part of the initial routine assessment procedure. The youth sample had a mean age of 15 years (SD 1.58) and was 84.1% female (n = 217). Youth participants were further categorized for analysis purposes based on the types of problems they displayed. Cutoff T-scores were used on the internalizing and externalizing scales of the CBCL parent reports, whereby T-scores of 70 or more on either scale was required to be categorized into the given problem group. The internalizing problems group represented 61.6% of the youth portion of the sample, while 31% were categorized into the externalizing problems group. The rest of the sample (7.4%) did not make the cut offs for either group and were excluded from concerned analyses. For those youth having CBCL forms completed by both parents, the mother’s version was retained for inclusion in the study, whom therefore represent 84.1% (n = 217) of the parent portion of the sample.
Measures
The ASEBA is a preliminary screening tool widely used to measure behaviour and to organize it according to appropriate norms based on age, sex, and informant, and is free from any form of clinician bias.25,26 Two forms from this set of www.TheCJP.ca
assessment tools were used in our study: the YSR completed by youths, and the CBCL completed by parents. Both the YSR and CBCL contain 2 parts: 2 broadband problem subscales (internalizing and externalizing problems) evaluated on 8 narrowband subscales and 3 competency subscales (activities, social skills, and academic competence). The validity and reliability of both the YSR and CBCL have been established.4
Statistical Analyses
Raw scores for the subscales were used in all analyses because they are more representative of the distribution and are preferred over T-scores for analyses of the CBCL and YSR.9 ICCs were calculated to investigate the parent‒child agreement across the activities, social skills, and academic subscales of both forms. Following Hallgreen’s27 descriptions of various common ICC variants, the 1-way random approach for ICC was used in our study. Specific guidelines for interpreting ICCs are not available, therefore Cohen’s conventions are used when other norms are unavailable.28 According to Cohen’s conventions29 for interpreting ICCs, below r = 0.40 are weak, between 0.40 and 0.60 are moderate, and above 0.60 are strong. ICC 95% confidence intervals were also calculated as suggested by Shrout and Fleiss30 when generalizing from a single rating to a mean rating of reliability, as is the case with our study’s methodology, to approximate the population value for these correlations.
Results
The means and standard deviations for the broadband problems and competencies scales are presented in Table 1 along with the sample’s demographic variables. Scores on the problem scales are above average, while scores on the competencies are below average. Table 2 describes the ICCs for each competency subscale. Only correlations for the activities and social subscales were significant at P < 0.001, with ICCs r = 0.46 and r = 0.34, respectively, indicating weak-to-moderate agreement between parents and youths on these competency subscales. Correlations for the academic subscale were nonsignificant. Sex differences were examined to evaluate whether ICCs would show the same degree and direction of agreement. Table 3 summarizes the parent‒child ICCs based on sex across all 3 competencies subscales. Correlations for the activities and social subscales were significant, with rs indicating weak agreement for females on the activities subscales and strong agreement for males. Agreement between informants on the social subscale was weak for both males and females. Agreement on the academic subscale was nonsignificant. Cross-informant agreement was further investigated according to age of the youth, whereby those aged 11 to 14 years were compared with adolescents aged 15 through 18 years. Table 4 describes the results for the ICCs separated by age groups. All ICCs on the activities and social subscales were statistically significant. Correlation coefficients indicate moderate agreement on the activities subscale and weak agreement on the social scale across age groups. Informant agreement on the academic subscale did not reach statistical significance. The Canadian Journal of Psychiatry, Vol 60, Supplement 1, February 2015 W S57
Chapter 6
Table 1 Demographic variables and YSR and CBCL scales n (%)
Variable
YSR youth-reported T-score (SD)
CBCL parent-reported T-score (SD)
Sex
80 (31)
Male
178 (69)
Female
15.00 (1.58)
Age, years, mean (SD) 11 to 14
89 (34.5)
15 to 18
169 (65.5)
Ethnicity
221 (85.7)
Caucasian Asian
8 (3.1)
African American
3 (1.2)
Latino
6 (2.3)
Native American
2 (0.8)
Other
10 (3.9)
Externalizing problems
80 (31)
62.92 (10.46)
65.15 (9.50)
Internalizing problems
159 (62)
66.57 (11.77)
71.22 (8.28)
Activities
41.13 (10.67)
40.99 (9.47)
Social
38.96 (9.43)
38.12 (8.41)
—
42.27 (9.22)
Competencies
Academic — T-scores are not calculated for the YSR academic subscale CBCL = Child Behaviour Checklist; YSR = Youth Self-Report
Table 2 ICC for competency scales Competencies
ICC
95% CI
F
df
P
Activities
0.46
0.35 to 0.60
2.76
200/201
Chapter 6
Parent–Youth Agreement on Self-Reported Competencies of Youth With Depressive and Suicidal Symptoms Valentin Mbekou, PhD1; Sasha MacNeil, BA2; Martin Gignac, MD, FRCPC3; Johanne Renaud, MD, MSc, FRCPC4 1
Psychologist, Standard Life Centre for Breakthroughs in Teen Depression and Suicide Prevention, Douglas Mental Health University Institute, Montréal, Québec.
2
Student, Psychology Department, Concordia University, Montréal, Québec; Research Assistant, Standard Life Centre for Breakthroughs in Teen Depression and Suicide Prevention, Douglas Mental Health University Institute, Montréal, Québec.
3
Child and Adolescent Psychiatrist, Philippe Pinel Institute, Montréal, Québec.
4
Child and Adolescent Psychiatrist, Standard Life Centre for Breakthroughs in Teen Depression and Suicide Prevention, Douglas Mental Health University Institute, Montréal, Québec; Medical Chief, McGill Group for Suicide Studies, McGill University, Montréal, Québec. Correspondence: Douglas Mental Health University Institute, 6875 Lasalle Boulevard, Montréal, QC H4H 1R3; [email protected].
Key Words: adolescence, depression, psychopathology, inter-informant agreement, risk factors, protective factors, competencies Received January 2014, revised, and accepted March 2014.
Objective: A multi-informant approach is often used in child psychiatry. The Achenbach System of Empirically Based Assessment uses this approach, gathering parent reports on the Child Behaviour Checklist (CBCL) and youth reports on the Youth Self-Report (YSR), which contain scales assessing both the child’s problems and competencies. Agreement between parent and youth perceptions of their competencies on these forms has not been studied to date. Method: Our study examined the parent–youth agreement of competencies on the CBCL and YSR from a sample of 258 parent–youth dyads referred to a specialized outpatient clinic for depressive and suicidal disorders. Intraclass correlation coefficients were calculated for all competency scales (activity, social, and academic), with further examinations based on youth’s sex, age, and type of problem. Results: Weak-to-moderate parent–youth agreements were reported on the activities and social subscales. For the activities subscale, boys’ ratings had a strong correlation with parents’ ratings, while it was weak for girls. Also, agreement on activities and social subscales was stronger for dyads with the youth presenting externalizing instead of internalizing problems.
open access
Conclusion: Agreement on competencies between parents and adolescents varied based on competency and adolescent sex, age, and type of problem. WWW
Accord parents–adolescents sur les compétences auto-déclarées d’adolescents présentant des symptômes dépressifs et suicidaires Objectif : Une approche multi-informant est souvent utilisée en pédopsychiatrie. Le Système d’évaluation empirique Achenbach utilise cette approche qui recueille les déclarations des parents à la Child Behaviour Checklist (CBCL) et celles des adolescents au Youth Self-Report (YSR), lesquels contiennent des échelles qui évaluent les problèmes et les compétences de l’enfant. L’accord entre les perceptions des parents et des adolescents relativement à leurs compétences à ces mesures n’a pas été étudié jusqu’ici. Méthode : Notre étude a examiné l’accord parents–adolescents sur les compétences à la CBCL et à l’YSR dans un échantillon de 258 dyades parents–adolescents adressé à une clinique ambulatoire spécialisée dans les troubles dépressifs et suicidaires. Les coefficients de corrélation intraclasse ont été calculés pour toutes les échelles de compétence (activité, sociale, et scolaire), et d’autres examens selon le sexe, l’âge, et le type de problème de l’adolescent. Résultats : Des accords parents–adolescents faibles à modérés ont été observés aux souséchelles d’activités et sociale. Pour la sous-échelle d’activités, les cotes des garçons étaient fortement corrélées à celles des parents, tandis que les corrélations étaient faibles pour les filles. Aussi, aux sous-échelles d’activités et sociale, l’accord était plus prononcé dans les dyades où l’adolescent présentait des problèmes d’externalisation plutôt que d’internalisation. Conclusion : L’accord sur les compétences entre les parents et les adolescents variait selon la compétence et le sexe, l’âge, et le type de problème.
www.TheCJP.ca
The Canadian Journal of Psychiatry, Vol 60, Supplement 1, February 2015 W S55
Chapter 6
I
n child psychiatry, it is common practice to consult multiple informants’ ratings during assessment,1 thereby gathering information from the youth, their parents, and, in some cases, their teachers. Each informant additional to the child is believed to contribute a new perspective on the child’s behaviour depending on the amount of time spent with the child, the individual’s closeness to the child, the situation of the observation, and the purpose of the assessment.2 The study of cross-informant agreement has been of interest to provide effective interventions based on a more complete conceptualization of the youth’s functioning.3
An important tool in child psychiatry is the ASEBA,4 which contains various questionnaires using a multi-informant approach to assess the youth’s functioning. Among the questionnaires comprising the ASEBA, youths provide their own perception of their personal experience on the YSR form while parents complete a comparable form, the CBCL, regarding their perceptions of their children. Research evaluating agreement across multiple informants has repeatedly shown differences between self-reported and parent-reported behaviours and problems in youths. Among clinically referred samples, it has been found that parents report as many or more externalizing problems than do the youths, likely due to the easily observable and public nature of associated behaviours.1,2,5 It has been suggested that adolescents are not the most reliable source for indicating externalizing problems,6 possibly because of a tendency to minimize the importance of their problems on behaviourrating checklists, or because of denial or lack of insight.7 Conversely, for internalizing problems parental reports tend to underestimate the severity, seeing as youth will report more internalizing problems.2,5 Internalizing problems, therefore, seem to be poorly recognized by parents due to their less observable and more private nature,1,8 and the youths are a better source of this kind of information. Cross-informant agreement is usually expected to be greater for externalizing problems than internalizing problems for this reason, yet mixed results have been found.5,8 Opposite patterns of crossinformant agreement have been observed among nonclinical youth samples; however, youths tend to self-report higher severity ratings and more problems of both kinds in comparison to their parents.1,5,8,9 These findings have been replicated across cultures,5 yet further examination within these patterns of agreement has shown differences among them based on youth characteristics, such as age and sex. Edelbrock et al10 found differences in parent–child agreement on a structured interview that were contingent with the age of the child, such that agreement was stronger among dyads with older youths (14 to 18 years) than those with younger youths (10 to 13 years). These results were also found using the ASEBA’s CBCL and YSR.11 Abbreviations ASEBA Achenbach System of Empirically Based Assessment CBCL
Child Behaviour Checklist
ICC
intraclass correlation coefficient
YSR
Youth Self-Report
S56 W La Revue canadienne de psychiatrie, vol 60, supplément 1, février 2015
Clinical Implications •
Clinicians should consider risk and protective factors in the shaping of interventions.
•
Parental reports of youth competencies can be used with weak-to-moderate reliability.
Limitations •
Our study is limited by the Berkson bias.
•
Competencies reported by parents on the CBCL may not reflect the youths’ actual competencies.
Most research and clinical work to date in the field of adolescent mental health, including the use of the CBCL and YSR, has focused on the identification and treatment of various forms of problems and psychopathologies.6 More recently, there has been growing interest for the role of protective factors and well-being in mental health outcomes and their integration into etiological models of mental health to provide a more complete understanding of functioning.12 Studies6,12,13 have shown the relevance of including well-being as a dimension in mental health models. The ASEBA tools contain both these dimensions, whereby youths are scored on both problem and competencies subscales. The CBCL and YSR contain 2 super-ordinate problem subscales to assess internalizing and externalizing problems, while further assessing 3 competency subscales (activities, social competence, and academics) to obtain an overarching total competence score.14 The activities subscale explores the youth’s implications in sports and nonsports‒related activities (how many and how implicated they are within these activities), as well as jobs and job performance. The social competence subscale examines involvement within groups and organizations; involvement with friend groups; behaviour with peers, siblings and family; and amount of time spent alone. The academic subscale evaluates school performance (failures, class rank), whether the youth has repeated grades, is in remedial class, or if other school problems are reported.14 Research has shown that the competencies explored by the CBCL and YSR are related to positive outcomes in adolescent psychopathology. More involvement in activities (sports, social, and academic) during high school has been associated with lower levels of externalizing symptoms, along with less substance use and behaviour disorders.15 Difficulties with peers and (or) teachers, preparing for examinations, reading and writing tasks, self-reliant school performance, and other school performance issues have been associated with elevated depression among both boys and girls.16 Effective social competence has been linked with reduced likelihood of internalizing problems.17 Therefore, the assessment of such competencies may provide information about outcomes beyond the present risk factors. The ASEBA assessment tools competency scales have not been studied as extensively as the problem scales. The few studies examining the competency scales have reported mixed results. Study of the competencies has found sex differences on the social competence and total competence www.LaRCP.ca
Parent–Youth Agreement on Self-Reported Competencies of Youth With Depressive and Suicidal Symptoms
scales18,19 and others have not.20,21 Age effects were also reported by some, whereby adolescents aged 17–18 years were found to be rated higher on the activity subscale and on total competence than younger adolescents,18 while an inverse relation has been reported by others.21 A few studies have examined cross-informant agreement (including reports from self, peers, parents, and teachers) of youth competencies.22‒24 A meta-analysis conducted by Renk and Phares3 reported a small correlation (r = 0.21) between self-reports and parental reports of social competence, with greater agreement among dyads with younger children than with older ones. These studies of cross-informant agreement have not, however, used the CBCL or YSR from the ASEBA. In light of the importance of well-being on mental health outcomes and the widespread use of the CBCL and YSR in child psychiatry, our study aimed to examine the agreement between parents and youths for youth competencies as reported on the CBCL and YSR. Only a few studies have examined this. Further investigation into youth characteristics that may affect agreement was also undertaken to evaluate whether the degree of agreement would change depending on the youth’s age, sex, and the types of problems they present.
Method Participants
All youths who were included in the study were referred from first- or second-line health clinics to the Depressive and Suicidal Disorders Outpatient Clinic (youth section) at the Douglas Mental Health University Institute because of depressive symptoms. Permission to access medical files of patients followed from 2006 to 2013 was granted through the Directorate of Professional and Hospital Services of the Douglas Mental Health University Institute. A total of 258 parent‒child dyads were included in our study. Upon entering the specialized outpatient clinic, the CBCL and YSR were administered to the parent(s) and the youth as part of the initial routine assessment procedure. The youth sample had a mean age of 15 years (SD 1.58) and was 84.1% female (n = 217). Youth participants were further categorized for analysis purposes based on the types of problems they displayed. Cutoff T-scores were used on the internalizing and externalizing scales of the CBCL parent reports, whereby T-scores of 70 or more on either scale was required to be categorized into the given problem group. The internalizing problems group represented 61.6% of the youth portion of the sample, while 31% were categorized into the externalizing problems group. The rest of the sample (7.4%) did not make the cut offs for either group and were excluded from concerned analyses. For those youth having CBCL forms completed by both parents, the mother’s version was retained for inclusion in the study, whom therefore represent 84.1% (n = 217) of the parent portion of the sample.
Measures
The ASEBA is a preliminary screening tool widely used to measure behaviour and to organize it according to appropriate norms based on age, sex, and informant, and is free from any form of clinician bias.25,26 Two forms from this set of www.TheCJP.ca
assessment tools were used in our study: the YSR completed by youths, and the CBCL completed by parents. Both the YSR and CBCL contain 2 parts: 2 broadband problem subscales (internalizing and externalizing problems) evaluated on 8 narrowband subscales and 3 competency subscales (activities, social skills, and academic competence). The validity and reliability of both the YSR and CBCL have been established.4
Statistical Analyses
Raw scores for the subscales were used in all analyses because they are more representative of the distribution and are preferred over T-scores for analyses of the CBCL and YSR.9 ICCs were calculated to investigate the parent‒child agreement across the activities, social skills, and academic subscales of both forms. Following Hallgreen’s27 descriptions of various common ICC variants, the 1-way random approach for ICC was used in our study. Specific guidelines for interpreting ICCs are not available, therefore Cohen’s conventions are used when other norms are unavailable.28 According to Cohen’s conventions29 for interpreting ICCs, below r = 0.40 are weak, between 0.40 and 0.60 are moderate, and above 0.60 are strong. ICC 95% confidence intervals were also calculated as suggested by Shrout and Fleiss30 when generalizing from a single rating to a mean rating of reliability, as is the case with our study’s methodology, to approximate the population value for these correlations.
Results
The means and standard deviations for the broadband problems and competencies scales are presented in Table 1 along with the sample’s demographic variables. Scores on the problem scales are above average, while scores on the competencies are below average. Table 2 describes the ICCs for each competency subscale. Only correlations for the activities and social subscales were significant at P < 0.001, with ICCs r = 0.46 and r = 0.34, respectively, indicating weak-to-moderate agreement between parents and youths on these competency subscales. Correlations for the academic subscale were nonsignificant. Sex differences were examined to evaluate whether ICCs would show the same degree and direction of agreement. Table 3 summarizes the parent‒child ICCs based on sex across all 3 competencies subscales. Correlations for the activities and social subscales were significant, with rs indicating weak agreement for females on the activities subscales and strong agreement for males. Agreement between informants on the social subscale was weak for both males and females. Agreement on the academic subscale was nonsignificant. Cross-informant agreement was further investigated according to age of the youth, whereby those aged 11 to 14 years were compared with adolescents aged 15 through 18 years. Table 4 describes the results for the ICCs separated by age groups. All ICCs on the activities and social subscales were statistically significant. Correlation coefficients indicate moderate agreement on the activities subscale and weak agreement on the social scale across age groups. Informant agreement on the academic subscale did not reach statistical significance. The Canadian Journal of Psychiatry, Vol 60, Supplement 1, February 2015 W S57
Chapter 6
Table 1 Demographic variables and YSR and CBCL scales n (%)
Variable
YSR youth-reported T-score (SD)
CBCL parent-reported T-score (SD)
Sex
80 (31)
Male
178 (69)
Female
15.00 (1.58)
Age, years, mean (SD) 11 to 14
89 (34.5)
15 to 18
169 (65.5)
Ethnicity
221 (85.7)
Caucasian Asian
8 (3.1)
African American
3 (1.2)
Latino
6 (2.3)
Native American
2 (0.8)
Other
10 (3.9)
Externalizing problems
80 (31)
62.92 (10.46)
65.15 (9.50)
Internalizing problems
159 (62)
66.57 (11.77)
71.22 (8.28)
Activities
41.13 (10.67)
40.99 (9.47)
Social
38.96 (9.43)
38.12 (8.41)
—
42.27 (9.22)
Competencies
Academic — T-scores are not calculated for the YSR academic subscale CBCL = Child Behaviour Checklist; YSR = Youth Self-Report
Table 2 ICC for competency scales Competencies
ICC
95% CI
F
df
P
Activities
0.46
0.35 to 0.60
2.76
200/201
