Research in Developmental Disabilities 38 (2015) 161–170

Contents lists available at ScienceDirect

Research in Developmental Disabilities

Elaboration, validation and standardization of the five to fifteen (FTF) questionnaire in a Danish population sample Rikke Lambek *, Anegen Trillingsgaard Department of Psychology, Aarhus University, Bartholins Alle´ 9, 8000 Aarhus C, Denmark

A R T I C L E I N F O

A B S T R A C T

Article history: Received 1 October 2014 Accepted 9 December 2014 Available online 12 January 2015

The five to fifteen (FTF) is a parent questionnaire developed to assess ADHD, its common comorbid conditions and associated problems in children and adolescents. The present study examined (1) the psychometric properties of scores on the new teacher version of the FTF, (2) competing models of the FTF subdomain structure and (3) the psychometric properties and utility of scores on the newly developed FTF impact questions. Parents (n = 4258) and teachers (n = 1298) of Danish children and adolescents (ages 5 to 17 years), selected using simple random sampling, completed the FTF. In the largest study of the FTF to date, parent and teacher scores had acceptable psychometric properties. The FTF subdomains were organized into six domains labelled cognitive skills, motor/perception, emotion/socialization/behaviour, attention, literacy skills and activity control and analysis of these domains may provide additional information when applying the FTF in the future. The impact questions yielded information above and beyond that provided by symptom count alone and appeared to increase the ability of the FTF to identify at risk children and adolescents. ß 2014 Elsevier Ltd. All rights reserved.

Keywords: Five to fifteen (FTF) questionnaire ADHD Teachers Impact

1. Introduction Neurodevelopmental disorders such as attention-deficit/hyperactivity disorder (ADHD) are associated with high rates of comorbidity and overlapping problems such as executive dysfunction, social skills deficits and language impairment (Jensen et al., 2001; Staikova, Gomes, Tartter, McCabe, & Halperin, 2013; Willcutt, Doyle, Nigg, Faraone, & Pennington, 2005). While several excellent questionnaires (Achenbach & Rescorla, 2001; DuPaul, Power, Anastopoulos, & Reid, 1998) assess some of these comorbidities/problems, the need for a scale that covers the whole range led a Nordic multidisciplinary research group to develop the five to fifteen (FTF) questionnaire which targets ADHD, its common comorbid conditions and associated problems in children and adolescents aged 5 to 15 years (Kadesjo¨ et al., 2004). The FTF questionnaire (available at www.5-15. org) has 181 items that can be endorsed as ‘‘does not apply’’ (0), ‘‘applies sometimes or to some extent’’ (1) or ‘‘definitely applies’’ (2). Items are arranged into eight domains covering motor skills, executive functions, perception, memory, language, learning competencies, social skills and emotional/behavioural problems. The domains can be further divided into 22 subdomains investigating gross and fine motor skills, attention, hyperactivity-impulsivity, hypoactivity, planning/ organising, perception of space, time, and body, as well as visual perception, memory, comprehension, speech, communication skills, reading/writing, math, general learning, coping skills during learning, social skills, internalizing and

* Corresponding author. Tel.: +45 87165815; fax: +45 87150201. E-mail address: [email protected] (R. Lambek). http://dx.doi.org/10.1016/j.ridd.2014.12.018 0891-4222/ß 2014 Elsevier Ltd. All rights reserved.

162

R. Lambek, A. Trillingsgaard / Research in Developmental Disabilities 38 (2015) 161–170

externalizing behaviour and obsessive–compulsive behaviour. The reliability and validity of the FTF domain and subdomain scores have been demonstrated by studies finding acceptable to good internal consistency, test–retest reliability, and interrater agreement (Kadesjo¨ et al., 2004) as well as significant associations with relevant scores from questionnaires (Bohlin & Janols, 2004) and performance-based measures (Korkman, Jaakkola, Ahlroth, Pesonen, & Turunen, 2004; Lind et al., 2010). The FTF is currently translated into 6 languages (more are pending) and used extensively in clinical practice and research. The FTF was originally developed as a parent questionnaire. However, the items pertain to aspects of child and adolescent functioning that should also be evident outside the family context (e.g., in school), and therefore teachers should be able to complete the FTF as well (Kadesjo¨ et al., 2004). The FTF has already been administered to teachers in a few previous studies and preliminary results suggest an association between parent and teacher ratings (Farooqi, Ha¨gglo¨f, & Serenius, 2013; Lindblad, Gillberg, & Fernell, 2011). As multi-informant ratings are generally recommended in the clinical assessment and hopefully should establish some level of reliability, an official teacher questionnaire was developed. This questionnaire was identical to the parent version, but ‘‘child’’ was substituted with ‘‘pupil’’ where relevant. The facture structure of the FTF subdomains has been examined in three previous studies using principal component analysis (PCA). One study found a two-factor solution, representing learning difficulties and socio-emotional problems in a population-based sample (Bohlin & Janols, 2004). Another population-based study resulted in one broad general development factor and three additional factors representing socio-emotional problems/control, cognition/motor function/ language, and communication/school learning (Beltra´n-Ortiz, de Barra, Franzani, Martinich, & Castillo, 2012). Finally, a study with a clinical sample with ADHD resulted in six factors including cognitive skills, motor/perception, emotion/socialization/ behaviour, attention, literacy skills, and activity control (Bruce, Thernlund, & Nettelbladt, 2006). Because of these inconsistent results the factor structure of the FTF subdomains deserves to be re-examined and using parent as well as teacher ratings. Mounting evidence suggests a discrepancy between the number of symptoms or problems reported and the degree of functional impairment endured. For instance, the rate of ADHD appears to decrease when diagnosis is based not solely on symptom count but also functional impairment (Gathje, Lewandowski, & Gordon, 2008), just as individuals not identified as cases by symptom count may suffer from significant impairment (Sibley et al., 2012). Inspired by this line of work, 10 impact questions were added to the FTF. The impact questions were formulated in general terms such as ‘‘Do problems with X interfere your child’s daily function’’ (parents) or ‘‘Do problems with X interfere with your pupil’s function in school’’ (teachers) to be rated as ‘‘Not at all’’ (0), ‘‘A little’’ (1), ‘‘Quite a lot’’ (2) or ‘‘A great deal’’ (3). Impact questions were placed immediately after domains, with the exception of the executive function domain, where separate impact questions were included after subdomains. This was done because the executive function domain included ADHD DSM-IV/5 congruent items as well as items pertaining to hypoactivity and planning/organising and recent studies suggest that neither hypoactivity, nor problems with planning/organising are symptomatic of ADHD, albeit more likely distinct but often cooccurring problems (Barkley, 2013; Lambek et al., 2011). Additionally, an upward extension of the age range to 16 and 17-year-olds was performed and a few items slightly elaborated to reflect a broader age range. Due to space constraints descriptive statistics for 16 to 17-year-olds are provided as supplementary data and only the major conclusions regarding the inclusion of older adolescents are reported in the article. The present study constitutes the largest study of the FTF questionnaire to date and is the first to examine the psychometric properties of FTF teacher scores, competing models of the FTF subdomain structure and the newly developed FTF impact questions. The study had three specific aims. The first aim was to assess the reliability of domain and subdomain scores. Second, the construct validity of the FTF questionnaire was investigated. Finally, the psychometric properties and utility of the impact scale scores were examined. 2. Method 2.1. Participants Statistics Denmark, a state institution under the Ministry of Economic Affairs and the Interior, was employed to select an age- and gender stratified simple random sample (approximately 1%) of the population of children between the ages of 5 and 17 years (N = 865,414) living in Denmark at the time of the study. After individuals with research protection and recent emigrants or deaths were excluded (n = 2185), parents of 9065 children were invited to participate. A total of 4263 parent questionnaires were returned, including questionnaires from parents of five sibling pairs; subsequently, one member of each sib-pair was randomly selected and excluded from the study. The majority of the remaining questionnaires (n = 4258) were completed by mothers (71%). The FTF sample (children) was mainly Caucasian of Danish descent (95%). Comparisons between the FTF sample and the background population indicated that differences were small (4% units in 8 out of 11 socioeconomic variables concerning labour force participation and income), albeit that the FTF sample was slightly underrepresented with respect to lower socioeconomic groups. Parents were encouraged to forward an invitation letter to the child’s primary teacher (see Section 2.2). A total of 1415 teacher questionnaires were returned. After excluding 115 teacher questionnaires without a corresponding parent questionnaire and one half of two sib-pairs, 1298 teacher questionnaires were included in the present study. The majority of teachers came from primary and lower secondary schools (77% public, 13% private, 2% special education, and 3% continuation), 3% came from kindergarten-equivalent facilities and 2% from upper secondary schools.

R. Lambek, A. Trillingsgaard / Research in Developmental Disabilities 38 (2015) 161–170

163

2.2. Procedure Invitation letters were sent by mail and included information about the study, the telephone number of the authors so they could be contacted if additional information was needed, a link to a website where parents, who wished to participate, could complete a short background questionnaire as well as the FTF, and a letter inviting the child’s primary teacher to fill out the teacher version of the FTF, also online. Parents were asked to forward the letter to the teacher, if they gave consent for him or her to participate. The teacher letter was similar to the parent version. Parents and teachers with no access to a computer were given the opportunity to receive and return a paper version of the FTF by mail. Parents received two reminders (one by mail and one by telephone) and were included in a drawing to win 2 IPADs. Parents and teachers were only asked questions from the FTF learning domain if the child was 8 years or older. Finally, parents were asked permission to be contacted again for follow-up, to which 90% of the sample consented. Again, using simple random sampling, an age- and gender stratified subsample was selected (n = 208 or 5% of the sample who agreed to follow-up) and readministered the FTF after one month (responders, n = 123). Teachers were not included in the follow-up, because all contact with teachers went through parents. In Denmark, questionnaire-based studies do not require ethics committee approval. Instead, the regional ethics committee was informed about the study taking place. The study was covered by the approval from the Data Protection Agency held by Statistics Denmark who removed all identifying information after data collection, thereby ensuring total anonymity for participants. 2.3. Additional instruments A brief questionnaire queried parents about background variables such as highest educational level achieved by mothers (a proxy for socioeconomic status [SES]) and child diagnostic status. 2.4. Main analytic strategy First, differences in mean domain and subdomain scores for parents and teachers were assessed using t-tests and the reliability of all domain and subdomain scores calculated using Cronbach’s alpha. Second, the construct validity of the FTF questionnaire was assessed using exploratory (EFA) and confirmatory factor analysis (CFA). The total sample was randomly split into two subsamples, one to use for model development using EFA based on parent ratings and the second to compare competing models using CFA based on parent and teacher ratings. Finally, the psychometric properties of the impact scale scores for parents and teachers were examined and the utility of parent-rated impact evaluated in relation to parent reported diagnoses of ADHD, autism spectrum disorder, and internalising/externalising disorder using hierarchical binary logistic regression analysis. The analysis was designed to determine if the impact questions would provide additional unique predictive validity in predicting a diagnosis after the contribution of control variables (age, gender, and SES) and the relevant FTF scales were statistically controlled for1. 3. Results Detailed descriptive statistics showing means and standard deviations for boys and girls for different age bands for all parent- and teacher-rated domains and subdomains are provided in Supplemental Tables 1 and 2. Descriptive statistics for FTF domains and subdomains are presented in Table 1. Internal consistency coefficients ranged from .853 to .961 for domains and from .649 to .961 for subdomains (parent FTF), and .907 to .968 for domains and .775 to .968 for subdomains (teacher FTF), suggesting acceptable to excellent internal consistency for parent and teacher ratings. The scale generated enough variability across subdomain means to suggest that there is no restriction of range and that individual differences are adequately reflected. There was a statistically significant difference between parent and teacher ratings in 7 of 8 domains and 13 of 22 subdomains. The mean differences were generally small, ranging from .010 to .041 (domains) and .000 to .093 (subdomains) with small effect sizes, ranging from .038 to .193 (domains) and .002 to .243 (subdomains). The presence of positive as well as and negative mean differences suggested no systematic bias. The latent structure of the 22 FTF subdomain scores was assessed in two phases. First, the data were randomly split into two subsamples and EFA was conducted on one subsample (n = 2128). Models that included one to ten factors, with an oblique (Geomin) rotation, were tested. Model parameters were estimated using robust maximum likelihood (MLR) using all available data. As suggested by Hoyle and Panter (1995) and Jackson et al. (2009) model fit was assessed using a range of fit statistics including the chi-square statistic, the root mean square error of approximation (RMSEA), the standardized root mean square residual (SRMR), the comparative fit index (CFI), and the Tucker–Lewis Index (TLI). A non-significant chisquare, RMSEA and SRMR values less than .08 and CFI and TLI values above .90 indicate acceptable fit. Parsimony was considered when selecting the best model. The fit statistics for the EFA models are presented in Table 2. The model with five factors was the simplest model that met the criteria for model fit for the CFI, TLI, RMSEA, and SRMR. Although the chi-square was large relative to the degrees of freedom, this should not lead to the rejection of the model due to

1

Due to small amounts of missing data very minor discrepancies from the original number of participants may appear in Section 3.

R. Lambek, A. Trillingsgaard / Research in Developmental Disabilities 38 (2015) 161–170

164

Table 1 Mean domain and subdomain scores, reliabilities, and differences in mean scores for parent and teacher FTF. Parent

FTF

a

Motor skills Gross motor skills Fine motor skills Executive functionsa Attention Hyperactive/impulsive Hypoactive Planning/organising Perceptiona Relation in space Time concepts Body perception Visual perception Memorya Languagea Comprehension Expressive language skills Communication Learninga Reading/writing Math General learning Coping in learning Social skillsa Emotional/behavioural problemsa Internalising Externalising Obsessive-compulsive a b

Teacher

Parent-teacher differences

M

SD

a

M

SD

a

.119 .117 .120 .238 .306 .181 .238 .226 .121 .084 .242 .102 .056 .179 .120 .142 .083 .134 .213 .238 .237 .134 .242 .109 .095 .101 .127 .056

.217 .261 .227 .329 .434 .318 .392 .405 .199 .201 .401 .234 .192 .268 .244 .301 .199 .325 .347 .425 .456 .325 .397 .229 .170 .207 .229 .173

.870 .870 .806 .943 .921 .876 .798 .753 .853 .649 .808 .682 .708 .961 .919 .820 .874 .784 .961 .918 .921 .841 .926 .943 .918 .845 .871 .803

.133 .152 .113 .240 .307 .153 .308 .189 .096 .080 .126 .091 .083 .175 .145 .178 .100 .157 .236 .255 .221 .220 .245 .146 .074 .091 .085 .045

.263 .322 .267 .361 .470 .316 .494 .410 .216 .236 .305 .253 .258 .327 .297 .365 .236 .379 .397 .455 .466 .428 .407 .277 .152 .192 .207 .173

.907 .878 .876 .952 .936 .895 .877 .824 .910 .775 .792 .780 .826 .968 .940 .875 .897 .849 .968 .923 .941 .882 .924 .953 .916 .831 .888 .849

Mean difference .018 .035 .000 .017 .019 .024 .093 .020 .014 .003 .085 .008 .030 .010 .033 .054 .023 .023 .030 .027 .011 .088 .012 .041 .017 .002 .038 .012

Cohen’s db .076 .112 .002 .056 .047 .075 .205 .050 .069 .014 .227 .032 .128 .038 .132 .158 .111 .065 .099 .082 .026 .243 .033 .193 .112 .011 .168 .061

t (df) 2.731 4.028 .053 2.022 1.691 2.690 7.404 1.788 2.489 .519 8.154 1.141 4.604 1.368 4.767 5.700 4.006 2.328 3.142 2.604 .818 7.688 1.053 6.949 4.019 .414 6.033 2.207

p (1291) (1282) (1289) (1297) (1297) (1296) (1297) (1290) (1295) (1285) (1289) (1290) (1290) (1294) (1296) (1295) (1295) (1294) (1003) (1002) (986) (1003) (1003) (1296) (1296) (1296) (1296) (1296)

.006 .000 .957 .043 .091 .007 .000 .074 .013 .604 .000 .254 .000 .172 .000 .000 .000 .020 .002 .009 .414 .000 .292 .000 .000 .679 .000 .027

Domains are italicized. Effect size small: Cohen’s d = .20 to .50, medium: d = .50 to .80, large: d = .80 and higher.

the increased power of the test with large samples (Tanaka, 1987). This model also produced admissible estimates that were clearly interpretable and therefore was judged to be the best model. The factor loadings and factor correlations are reported in Table 3. Second, using the other subsample (n = 2128) the model that was considered the best EFA model was tested along with four competing models using CFA (Gerbing & Hamilton, 1996). The same criteria for the assessment of model fit were used as described previously. The alternative models were based on the results from factor analyses by Beltra´n-Ortiz et al. (2012), Bohlin and Janols (2004), Bruce et al. (2006) that tested the dimensionality of the FTF. The structure implied by the organization of the domains and subdomains of the FTF (Kadesjo¨ et al., 2004) was also specified and labelled the ‘‘FTF model’’. Simple structure models were specified by including only the highest loading subdomain for each factor. When the model included a single subdomain loading on a factor the model was identified by fixing the loading at a value determined by the reliability of the subdomain as suggested by Jo¨reskog and So¨rbom (1982). The structure of the alternative models is presented in Table 4.

Table 2 Fit statistics for EFA models of the FTF subdomains for parent responses. Number of factors 1 2 3 4 5 6 7 8 9 10

Fit statistics

x2 (df) p 2594.350 191.058 1391.276 967.429 762.874 405.220 405.220 291.638 171.252 119.628

(209) .00 (188) .00 (168) .00 (149) .00 (131) .00 (98) .00 (98) .00 (83) .00 (69) .00 (56) .00

CFI

TLI

RMSEA (90% CI)

SRMR

.788 .847 .891 .927 .944 .964 .973 .981 .991 .994

.765 .812 .850 .887 .901 .928 .936 .948 .970 .977

.073 .066 .058 .051 .048 .041 .038 .034 .026 .023

.067 .050 .034 .028 .022 .018 .014 .011 .008 .007

(.071–.076) (.063–.068) (.056–.061) (.048–.054) (.044–.051) (.037–.044) (.035–.042) (.030–.039) (.021–.031) (.017–.029)

x2 = Robust WLSMV chi-square; df = degrees of freedom; CFI = comparative fit indices; RMSEA = root mean square error of approximation; SRMR = the standardized root mean square residual; TLI = Tucker Lewis indices.

R. Lambek, A. Trillingsgaard / Research in Developmental Disabilities 38 (2015) 161–170

165

Table 3 Parent factor loadings and factor correlations from the five factor EFA solution of the FTF subdomains. FTF subdomain

Factora 1

Gross motor skills Fine motor skills Attention Hyperactive/impulsive Hypoactive Planning/organising Relation in space Time concepts Body perception Visual perception Memory Comprehension Expressive language skills Communication Reading/writing Math General learning Coping in learning Social skills Internalising Externalising Obsessive-compulsive Factor 1 Factor 2 Factor 3 Factor 4 Factor 5

2

3

4

5

.666* .614* .770*

.321* .541*

.365*

.675* .422* .448* .518* .548* .546* .327*

.611* .699* .706* .594* .522* .562* .755* .371*

.375* .375* .559*

.765* .671* .821* .660* 1.00 .426* .606* .193* .607*

1.00 .563* .344* .515*

1.00 .364* .547*

1.00 .438*

1.00

a

Only loadings > .30 shown and highest loading variable on each factor in bold. * p < .05.

The CFA supported a factor solution similar to the one proposed by Bruce et al. (2006), with 6 factors, and this was the case for the data from parents as well as teachers (Table 5). The fit statistics are similar for the Bruce et al., 6-factor solution and the 5-factor solution from the EFA, but the Bruce model is marginally better, so it has to be considered the ‘‘best’’ model. The factor loadings and factor correlations for the 6-factor CFA model are reported in Table 6. Descriptive statistics for impact scores are presented in Table 7. Parents and teachers differed significantly in 9 out of 10 impact ratings, albeit differences were generally small as indicated by small mean differences (ranging from .001 to .188) and small effect sizes (ranging from .002 to .297). The presence of primarily negative mean differences suggested that teachers generally rated somewhat higher impact than parents. Test–retest correlation values for parent-rated impact were modest to high, positive and statistically significant (rImpact of motor problems = .913, n = 122, p < .001; rImpact of inattention, overactivity or impulsivity = .592, n = 120, p < .001; rImpact of hypoactivity = .714, n = 121, p < .001; rImpact of planning/organising problems = .667, n = 121, p < 001; rImpact of perception problems = .724, n = 118, p < .001; rImpact of memory problems = .564, n = 121, p < .001; rImpact of language problems = .732, n = 119, p < .001; rImpact of learning problems = .703, n = 92, p < .001; rImpact of social problems = .772, n = 122, p < .001; rImpact of emotional/behavioural problems = .648, n = 122, p < .001). This suggests acceptable to high degree of retest stability for parent-reported impact on the FTF after one month. For all diagnoses (ADHD, autism spectrum disorder and internalizing/externalizing disorder) there were higher than expected rates of impact ratings greater than the ‘‘Not at all’’ category (see Supplemental Table 3). Hierarchical binary logistic regression analysis was carried out using Mplus 7 (Muthe´n & Muthe´n, 2013) with MLR estimation in order to assess the predictive validity of three impact questions. The analysis was designed to determine if the impact questions would provide additional unique predictive utility in predicting a parent-reported diagnosis of ADHD, autism spectrum disorder or internalising/externalising disorder after the contribution of control variables (age, gender, and SES) and the relevant FTF scales were statistically controlled for. Three separate analyses were conducted. In the first analysis the variable representing a diagnosis of ADHD was the dependent variable and age, gender, SES and the mean score on the FTF ADHD scale were entered in the first block. The 4-category ADHD impact question was dummy coded using ‘‘Not at all’’ as the reference category, resulting in three variables representing ‘‘A little’’, ‘‘Quite a lot’’ and ‘‘A great deal’’. The three dummy-coded variables were entered in the second block. The estimates from the model are odds ratios and indicate the expected increase in the likelihood of a diagnosis of ADHD for each level of the impact question compared to the ‘‘Not at all’’ category while controlling for the effects of the variables in the first block. This analysis was replicated for the diagnosis of autism spectrum disorder and internalising/externalising disorder using the relevant mean scores on FTF domains (social skills mean of items and emotional/behavioural problems mean of items) and impact questions. An impact rating of ‘‘A little’’ on the ADHD

166

R. Lambek, A. Trillingsgaard / Research in Developmental Disabilities 38 (2015) 161–170

Table 4 Item mapping for alternative factor models of the FTF subdomains. FTF subdomain

Gross motor skills Fine motor skills Attention Hyperactive/impulsive Hypoactive Planning/organising Relation in space Time concepts Body perception Visual perception Memory Comprehension Expressive language skills Communication Reading/writing Math General learning Coping in learning Social skills Internalising Externalising Obsessive-compulsive

Model Bohlin & Janols 2-factor model

Bruce et al. 6-factor model

Beltra´n-Ortiz et al. 4-factor model

FTF 8-factor model

LD LD LD SEP LD LD LD LD SEP/LD LD LD LD LD LD LD LD LD LD SEP SEP SEP SEP

MP MP A AC A A MP CS MP MP CS CS CS CS LS LS CS A ESB ESB ESB ESB

CML CML GD GD GD GD CML CML CML CML CML CSL CSL CSL CSL GD CSL GD SEPC SEPC SEPC SEPC

MS MS EF EF EF EF P P P P M L L L LE LE LE LE SS EBP EBP EBP

Bohlin & Janols LD = learning difficulties, SEP = socio-emotional problems. Bruce, Thernlund, & Nettelbladt A = attention, CS = cognitive skills, MP = motor/ perception, ESB = emotion/socialization/behaviour, AC = activity control, LS = literacy skills. Beltra´n-Ortiz et al. GD = general development, SEPC = socioemotional problems/control, CML = cognition/motor function/language, CSL = communication/school learning. FTF model MS = motor skills, EF = executive functions, P = perception, M = memory, L = language, LE = learning, SS = social skills, EBP = emotional/behavioural problems.

equivalent impact question increased the probability of having an ADHD diagnosis by 8 times compared to those children who got a ‘‘Not at all’’ rating. Odds-ratios tended to increase with impact severity level (Table 8). For the ADHD analysis the addition of the different impact levels significantly improved the model (Dx2(3) = 37.98, p < .01), and this was the same for autism spectrum disorder (Dx2(3) = 27.66, p < .01) and internalising/externalising disorder (Dx2(3) = 37.98, p < .01). This suggests that the impact ratings provide information above and beyond the information attained by counting endorsed items on the scales. 4. Discussion Based on the largest sample to date, the psychometric properties of scores on the elaborated FTF, which includes a teacher version and newly developed impact questions, were examined. The parent and teacher FTF scores were comparable and had acceptable psychometric properties. The performance of the scores remained acceptable with the inclusion of 16 to 17-yearolds. The FTF subdomains were found to be organized into six domains labelled cognitive skills, motor/perception, emotion/ socialization/behaviour, attention, literacy skills, and activity control and this was the case for the data from parents as well

Table 5 Fit statistics for CFA models of the FTF subdomains for parent and teacher responses. Model

Rater

x2 (df) p

Bruce et al. 6-factor model

Parent Teacher Parent Teacher Parent Teacher Parent Teacher Parent Teacher

1239.806 1078.224 2144.041 1664.190 1676.498 1342.035 1281.063 1139.001 1293.291 1139.960

Bohlin & Janols 2-factor model Beltra´n-Ortiz, et al. 4-factor model FTF 8-factor model EFA 5-factor model

(195) (195) (207) (207) (203) (203) (183) (183) (200) (200)

.00 .00 .00 .00 .00 .00 .00 .00 .00 .00

CFI

TLI

RMSEA

SRMR

.915 .890 .843 .818 .880 .858 .911 .881 .911 .883

.900 .869 .825 .797 .864 .838 .887 .849 .897 .864

.050 .059 .066 .074 .058 .066 .053 .063 .051 .060

.043 .052 .057 .065 .055 .061 .046 .056 .044 .055

(.048–.053) (.056–.063) (.064–.069) (.070–.077) (.056–.061) (.062–.069) (.050–.056) (.060–.067) (.048–.053) (.057–.064)

x2 = Robust WLSMV chi-square; df = degrees of freedom; CFI = comparative fit indices; EFA = exploratory factor analysis; RMSEA = root mean square error of approximation; SRMR = the standardized root mean square residual; TLI = Tucker Lewis indices.

R. Lambek, A. Trillingsgaard / Research in Developmental Disabilities 38 (2015) 161–170

167

Table 6 Factor loadings and factor correlations from the Bruce et al. six factor CFA solution for parent responses. Factora

FTF subdomain

A Gross motor skills Fine motor skills Attention Hyperactive/impulsive Hypoactive Planning/organising Relation in space Time concepts Body perception Visual perception Memory Comprehension Expressive language skills Communication Reading/writing Math General learning Coping in learning Social skills Internalising Externalising Obsessive-compulsive A CS MP ESB AC LS

CS

MP

ESB

AC

LS

.725 .804 .904 .894 .789 .820 .770 .614 .744 .751 .870 .859 .770 .779 .744 .729 .884 .913 .920 .689 .772 .698 1.00 .847 .742 .804 .696 .908

1.00 .828 .793 .625 .953

1.00 .764 .589 .780

1.00 .751 .697

1.00 .563

1.00

A = attention, CS = cognitive skills, MP = motor/perception, ESB = emotion/socialization/behaviour, AC = activity control, LS = literacy skills. a All factor loadings and correlations were significant, p < .01.

Table 7 Mean impact scores and differences in mean scores for parents and teachers. Impact item

Parent

Motor skills Attention/hyperactivity- impulsivity Hypoactivity Planning/organising Perception Memory Language Learning Social skills Emotional/behavioural problems a

Teacher

Parent-teacher differences

n

M

SD

n

M

SD

4248 4250 4250 4250 4248 4247 4251 3138 4251 4252

.085 .218 .201 .214 .061 .171 .119 .301 .181 .112

.339 .536 .501 .519 .292 .459 .409 .642 .513 .408

1287 1296 1298 1292 1290 1291 1296 1006 1298 1297

.132 .309 .369 .250 .107 .266 .201 .401 .259 .114

.413 .638 .672 .580 .375 .582 .537 .735 .591 .399

Mean difference .0428 .0934 .1880 .0495 .0473 .0987 .0772 .1175 .0779 .0008

Cohen’s da .106 .152 .297 .086 .126 .176 .157 .178 .138 .002

t (df) 3.806 5.466 10.715 3.082 4.518 6.328 5.657 5.648 4.955 .060

p (1285) (1295) (1297) (1291) (1288) (1286) (1294) (1003) (1295) (1295)

.000 .000 .000 .002 .000 .000 .000 .000 .000 .952

Effect size small: Cohen’s d = .20 to .50, medium: d = .50 to .80, large: d = .80 and higher.

Table 8 Odds ratios (95% confidence intervals) for FTF impact scales predicting ADHD, autism spectrum disorder and internalising/externalising disorder based on parent ratings. Predictor

ADHD (n = 82)

Autism spectrum disorder (n = 43)

Internalising/externalising disorder (n = 45)

Gender Age SES FTF scale A little Quite a lot A great deal

.593 (.320–1.101) 1.274*** (1.165–1.395) .871 (.683–1.111) 9.959*** (3.944–25.148) 8.062*** (2.528–25.711) 25.273*** (6.170–103.515) 38.141*** (7.138 –203.787)

.364* (.162–.818) 1.120 (.992–1.264) 1.416* (1.025–1.955) 26.728*** (9.052–78.924) 13.643** (2.462–75.600) 22.739** (3.260–158.608) 19.751** (2.313–168.644)

1.081 (.562–2.079) 1.365*** (1.225–1.521) 1.185 (.915–1.535) 8.966*** (2.813–28.577) 6.162*** (2.371–16.014) 19.749*** (5.745–67.885) 11.989*** (2.672–53.797)

* p < .05. ** p < .01. *** p < .001.

168

R. Lambek, A. Trillingsgaard / Research in Developmental Disabilities 38 (2015) 161–170

as teachers. The impact questions yielded information above and beyond that provided by symptom count alone. These results will be discussed below. 4.1. Reliability of the FTF Domains and subdomains from the parent and teacher versions of the FTF were found to have acceptable to excellent internal consistency with Cronbach’s alpha coefficients consistent with those reported for the original parent version (Beltra´n-Ortiz et al., 2012; Kadesjo¨ et al., 2004). Parent and teacher ratings differed significantly on the majority of domains and subdomains, although differences were generally small (average mean difference = .011; SD = .035). In one previous study moderate correlations were found between parent and teacher scores on FTF subdomains (Farooqi et al., 2013). In another study there was 86% to 89% agreement between parent and teacher ratings on FTF domains (Lindblad et al., 2011). Together with the results from the present study, this indicates that the parent and teacher versions of the FTF have enough common variance to suggest that they are measuring similar aspects of child functioning, but differ enough to merit the collection of ratings from the different respondents; that is parents and teachers appear to provide unique knowledge about the child. 4.2. Construct validity of the FTF The EFA-derived model specified 5 correlated factors labelled attention/organisation, motor function/perception, general cognition, hyperactivity/impulsivity, and emotional/social function. The CFA supported a factor solution similar to the one proposed by Bruce et al. (2006) with 6 correlated factors labelled attention, motor/perception, cognitive skills, activity control, emotion/socialization/behaviour, and literacy skills. The fit statistics and structures for these two models were quite similar. In the EFA-derived model reading/writing, math, time concepts, memory, comprehension, expressive language skills, communication, and general learning all loaded together on a common factor, whereas results from the CFA suggested that these subdomains formed two separate albeit highly correlated factors. Despite fit statistics, such as the RMSEA, rewarding more parsimonious models, the more complex Bruce et al. model had better fit statistics than the simpler EFAderived model so the Bruce et al. model should be considered the better model. Overall, the findings from the factor analyses suggested that the 22 FTF subdomains are organized into somewhat different (and fewer) domains than currently implied by the ‘‘FTF model’’. It is possible that some factors emerged because some behaviours are not easily teased apart by respondents, for instance it may be difficult to separate planning/organising from coping skills during learning or perception of time from memory. However, the domains correspond quite well with current research within the field of child psychopathology. For instance, attention, hypoactivity, planning/organization, and coping in learning loaded together on the attention factor which is in line with research finding inattention, sluggish cognitive tempo (i.e., hypoactivity), executive dysfunction, and poor academic achievement to be highly related (Langberg, Dvorsky, & Evans, 2013; Wa˚hlstedt & Bohlin, 2010). Hyperactivity-impulsivity constituted a factor of its own and may be more related with aspects of functioning not currently covered by the FTF (e.g., reward-related processes) (Willcutt et al., 2012). The emotion/socialization/behaviour factor comprised social skills, internalising behaviour, externalising behaviour, and obsessive-compulsive behaviour. Children with lower social competence in early childhood have been found to exhibit more externalizing and internalising behaviour in late childhood and early adolescence (Bornstein, Hahn, & Haynes, 2010). Consequently, social skills and behavioural adjustment appear to be closely related. Fine motor skills, gross motor skills, perception of space and body, and visual perception loaded on the motor/perception factor, which corresponds with research suggesting motor function and perception to be interrelated throughout development (Adolph & Joh, 2007). A relationship has also been proposed between reading/writing and math (literacy skills) (Davis et al., 2014). Finally, the fact that time concepts, memory, all language subdomains, and general learning loaded together suggests the presence of a broad cognitive skills factor. The 6-factor solution has now been replicated in a clinical as well as a population-based sample, using different factor analytic methods (PCA and CFA) and with parent as well as teacher data. Together these results support adding a scoring algorithm to the FTF by which these six domains can be assessed. 4.3. Utility of impact questions Teachers tended to report slightly more impact than parents, although differences were generally small. The largest differences between parent and teacher impact ratings were found on domains (e.g., hypoactivity and learning competencies) where the respondents would be expected to contribute with different perspectives. For instance, it has been suggested that hypoactivity is more readily observable in school during structured or effortful tasks (Jacobson et al., 2012); as a consequence the impact of hypoactivity may be most apparent in a school setting. In a similar vein, the impact of learning problems should be more noticeable in a learning environment such as the school. The impact questions also had acceptable to high degree of test-retest stability (parents). Taken together this suggests that the impact scores have acceptable psychometric properties. When the FTF impact questions from the ADHD, social skills and emotional/behavioural problems scales were evaluated in relation to parent-reported diagnoses of ADHD, autism spectrum disorder and internalising/externalising disorder, respectively, any parent-rated impact (i.e., a rating of ‘‘A little’’, ‘‘Quite a lot’’ or ‘‘A great deal’’) on a given impact question

R. Lambek, A. Trillingsgaard / Research in Developmental Disabilities 38 (2015) 161–170

169

increased the probability of having a parent-reported diagnosis of the equivalent type of child psychopathology. Consequently, if a parent rated that inattention, hyperactivity or impulsivity had ‘‘A great deal’’ of impact on the child’s daily function, then the probability of that child having an ADHD diagnosis increased by 38% compared to those children who got a ‘‘Not at all’’ rating. This was after the contribution of frequency of behaviour was statistically removed; consequently this indicates that assessment of impact provides additional and important information above and beyond that provided by counting endorsed items. The fact that adding impact questions to the FTF improved the explanatory value of the FTF model, attests to the utility of the impact questions. 4.4. Limitations and methodological considerations Teachers were recruited through parents. As it cannot be ascertained how many parents invited the child’s teacher to participate, the teacher response rate cannot be calculated. Furthermore, it is possible that the teacher sample would have been larger, had the parents not had to be actively involved in delivering the invitation letters to teachers. This is the first study to examine the psychometric properties of scores on the teacher version of the FTF and additional studies are needed to replicate and extend the findings. Parents reported impact as well as the diagnostic status of the child in the impact utility analyses. Consequently, an informant effect may have inflated the strength of the association between variables. Ideally we would have liked to include teacher impact ratings in these analyses, albeit as only around one third of the children whose parents reported their diagnostic status, had a corresponding teacher questionnaire, it was not deemed feasible to include teacher ratings. Future studies should attempt to further validate the FTF impact questions based on ratings from different informants. The sample was slightly underrepresented with respect to lower socioeconomic groups which should be taken into consideration when interpreting the results. 4.5. Conclusions and practical implications Scores on the FTF displayed acceptable psychometric properties and continued to do so with the inclusion of 16–17-yearolds, suggesting that the FTF can be used with children older than 15 years. The subdomains appeared to be organized into somewhat different (and fewer) domains than currently implied by the ‘‘FTF model’’ and analysis of these domains may provide additional information when applying the FTF in the future. The newly developed impact questions yielded information above and beyond that provided by symptom count alone, thus the impact questions appeared to have substantial utility as they may help identify at risk children and adolescents. Acknowledgments We are extremely grateful to all the families and teachers who took part in this study and to the FTF group for their encouragement and support. The study was funded by the TRYG Foundation (7-11-1204). Appendix A. Supplementary data Supplementary data associated with this article can be found, in the online version, at http://dx.doi.org/10.1016/j.ridd.2014. 12.018. References Achenbach, T. M., & Rescorla, L. A. (2001). Manual for the ASEBA school-age forms & profiles. Burlington, VT: University of Vermont, Research Center for Children, Youth, & Families. Adolph, K. E., & Joh, A. S. (2007). Motor development: How infants get into the act. In A. Slater & M. Lewis (Eds.), Introduction to infant development (2nd ed., pp. 63–80). Oxford: Oxford University Press. Barkley, R. A. (2013). Distinguishing sluggish cognitive tempo from ADHD in children and adolescents: Executive functioning, impairment, and comorbidity. Journal of Clinical Child and Adolescent Psychology, 42(2), 161–173. Beltra´n-Ortiz, M. F., de Barra, H. T., Franzani, P., Martinich, C., & Castillo, R. D. (2012). El cuestionario five to fifteen (FTF) para una evaluacio´n integral del desarrollo: Propiedades psicome´tricas y caracterizacio´n de muestra de nin˜os chilenos. Terapia Psicolo´gica, 30(3), 31–47. Bohlin, G., & Janols, L. O. (2004). Behavioural problems and psychiatric symptoms in 5–13 year-old swedish children—A comparison of parent ratings on the FTF (five to fifteen) with the ratings on CBCL (child behavior checklist). European Child & Adolescent Psychiatry, 13(Suppl. 3), 14–22. Bornstein, M. H., Hahn, C., & Haynes, O. M. (2010). Social competence, externalizing, and internalizing behavioral adjustment from early childhood through early adolescence: Developmental cascades. Development and Psychopathology, 22(4), 717–735. Bruce, B., Thernlund, G., & Nettelbladt, U. (2006). ADHD and language impairment: A study of the parent questionnaire FTF (five to fifteen). European Child & Adolescent Psychiatry, 15(1), 52–60. Davis, O. S. P., Band, G., Pirinen, M., Haworth, C. M. A., Meaburn, E. L., Kovas, Y., et al. (2014). The correlation between reading and mathematics ability at age twelve has a substantial genetic component. Nature Communications, 5, 4204. DuPaul, G. J., Power, T. J., Anastopoulos, A. D., & Reid, R. (1998). ADHD rating Scale—IV: Checklists, norms, and clinical interpretation. New York, NY: Guilford Press. Farooqi, A., Ha¨gglo¨f, B., & Serenius, F. (2013). Behaviours related to executive functions and learning skills at 11 years of age after extremely preterm birth: A Swedish national prospective follow-up study. Acta Paediatrica, 102(6), 625–634. Gathje, R. A., Lewandowski, L. J., & Gordon, M. (2008). The role of impairment in the diagnosis of ADHD. Journal of Attention Disorders, 11(5), 529–537. Gerbing, D. W., & Hamilton, J. G. (1996). Viability of exploratory factor analysis as a precursor to confirmatory factor analysis. Structural Equation Modeling, 3(1), 62–72.

170

R. Lambek, A. Trillingsgaard / Research in Developmental Disabilities 38 (2015) 161–170

Hoyle, R. H., & Panter, A. T. (1995). Writing about structural equation models structural equation modeling: Concepts, issues, and applications. Thousand Oaks, CA: Sage Publications Inc.. Jackson, D. L., Gillaspy, J. A., Jr., & Purc-Stephenson, R. (2009). Reporting practices in confirmatory factor analysis: An overview and some recommendations. Psychological Methods, 14(1), 6–23. Jacobson, L. A., Murphy-Bowman, S. C., Pritchard, A. E., Tart-Zelvin, A., Zabel, T. A., & Mahone, E. M. (2012). Factor structure of a sluggish cognitive tempo scale in clinically-referred children. Journal of Abnormal Child Psychology, 40(8), 1327–1337. Jensen, P. S., Hinshaw, S. P., Kraemer, H. C., Lenora, N., Newcorn, J. H., Abikoff, H. B., et al. (2001). ADHD comorbidity findings from the MTA study: Comparing comorbid subgroups. Journal of the American Academy of Child & Adolescent Psychiatry, 40(2), 147–158. Jo¨reskog, K. G., & So¨rbom, D. (1982). Recent developments in structural equation modeling. Journal of Marketing Research, 19(4), 404–416. Kadesjo¨, B., Janols, L., Korkman, M., Mickelsson, K., Strand, G., Trillingsgaard, A., et al. (2004). The FTF (five to fifteen): The development of a parent questionnaire for the assessment of ADHD and comorbid conditions. European Child & Adolescent Psychiatry, 13(Suppl. 3), 3–13. Korkman, M., Jaakkola, M., Ahlroth, A., Pesonen, A., & Turunen, M. (2004). Screening of developmental disorders in five-year-olds using the FTF (five to fifteen) questionnaire: A validation study. European Child & Adolescent Psychiatry, 13(Suppl. 3), 31–38. Lambek, R., Tannock, R., Dalsgaard, S., Trillingsgaard, A., Damm, D., & Thomsen, P. H. (2011). Executive dysfunction in school-age children with ADHD. Journal of Attention Disorders, 15(8), 646–655. Langberg, J. M., Dvorsky, M. R., & Evans, S. W. (2013). What specific facets of executive function are associated with academic functioning in youth with attentiondeficit/hyperactivity disorder? Journal of Abnormal Child Psychology, 41(7), 1145–1159. Lind, A., Haataja, L., Rautava, L., Va¨liaho, A., Lehtonen, L., Lapinleimu, H., et al. (2010). Relations between brain volumes, neuropsychological assessment and parental questionnaire in prematurely born children. European Child & Adolescent Psychiatry, 19(5), 407–417. Lindblad, I., Gillberg, C., & Fernell, E. (2011). ADHD and other associated developmental problems in children with mild mental retardation. The use of the ‘‘Five-tofifteen’’ questionnaire in a population-based sample. Research in Developmental Disabilities, 32(6), 2805–2809. Muthe´n, L. K., & Muthe´n, B. O. (2013). Mplus user’s guide (7th ed.). Los Angeles, CA: Muthe´n & Muthe´n. Sibley, M. H., Pelham, W. E., Jr., Molina, B. S. G., Gnagy, E. M., Waxmonsky, J. G., Waschbusch, D. A., et al. (2012). When diagnosing ADHD in young adults emphasize informant reports, DSM items, and impairment. Journal of Consulting and Clinical Psychology, 80(6), 1052–1061. Staikova, E., Gomes, H., Tartter, V., McCabe, A., & Halperin, J. M. (2013). Pragmatic deficits and social impairment in children with ADHD. Journal of Child Psychology and Psychiatry, 54(12), 1275–1283. Tanaka, J. S. (1987). How big is big enough?.: Sample size and goodness of fit in structural equation models with latent variables. Child Development, 58(1), 134–146. Wa˚hlstedt, C., & Bohlin, G. (2010). DSM-IV-defined inattention and sluggish cognitive tempo: Independent and interactive relations to neuropsychological factors and comorbidity. Child Neuropsychology, 16(4), 350–365. Willcutt, E. G., Doyle, A. E., Nigg, J. T., Faraone, S. V., & Pennington, B. F. (2005). Validity of the executive function theory of attention-deficit/hyperactivity disorder: A meta-analytic review. Biological Psychiatry, 57(11), 1336–1346. Willcutt, E. G., Nigg, J. T., Pennington, B. F., Solanto, M. V., Rohde, L. A., Tannock, R., et al. (2012). Validity of DSM-IV attention deficit/hyperactivity disorder symptom dimensions and subtypes. Journal of Abnormal Psychology, 121(4), 991–1010.