bs_bs_banner
doi:10.1111/jpc.12784
ORIGINAL ARTICLE
Predictive validity of the Italian parental questionnaire for developmental evaluation at age 4 (QS4-G) Francesca Bevilacqua,1 Patrizia Giannantoni,2 Patrizio Pasqualetti,3,4 Vincenzo M di Ciommo,5 Maria Franca Coletti,1 Lucilla Ravà,5 Maria Cristina Caselli6 and Anna Maria Dall’Oglio1 1
Unit of Clinical Psychology, Department of Neuroscience and Neurorehabilitation, 5Unit of Epidemiology, Bambino Gesù Children’s Hospital – IRCCS, Department of Statistics, Sapienza University of Rome, 3Service of Medical Statistics and Information Technology, Fatebenefratelli Foundation for Health Research and Education, AFaR Division, 4Language and Communication Across Modalities Laboratory (LACAM), Institute of Cognitive Sciences and Technologies (ISTC-CNR) and 6Institute of Cognitive Sciences and Technologies, National Research Council, Rome, Italy 2
Aim: To examine whether the results at 4 years of age of the developmental questionnaire QS4-G can predict the outcome of cognitive, neuropsychological and academic abilities 4–6 years later. The QS4-G is a validated parental questionnaire designed for the screening and surveillance of the neuropsychological and behavioural developmental status of 4-year-olds (93 questions). Methods: Longitudinal prospective study on a subsample of the QS4-G validation original sample was conducted. According to previous results, the sample was divided into two groups: ‘at risk’ and ‘not at risk’. Sensitivity, specificity, accuracy and likelihood ratios were assessed and referred to outcomes. Results: Thirty-five children were classified as ‘not at risk’ and 16 as ‘at risk’. There were significant associations between past QS4-G score and cognitive, neuropsychological and academic abilities 4–6 years later. With the same cut-off identified at the first cross-sectional study, sensitivity and specificity for difficulties in cognitive development were 90% and 83% while in the neuropsychological abilities 62% and 90%, respectively. A lower predictive validity was found for difficulties in academic abilities (sensitivity 43%, specificity 86%). QS4-G specific area scores showed significant correlations with related academic tests at follow-up (rho range: 0.404–0.565, P < 0.005). Conclusions: QS4-G shows good predictive validity for cognitive development and neuropsychological abilities. The risk of false negatives for academic abilities can be reduced by analysing the specific area results of QS4-G, which show good correlations with related tests at follow-up. Key words:
neuropsychological development; parental questionnaire; predictive validity.
What is already known on this topic
What this paper adds
1 QS4-G is a parental questionnaire designed to evaluate neuropsychological and behavioural developmental status of 4-year-old children. 2 QS4-G showed good reliability, internal consistency and discriminant validity. 3 It also showed good concurrent validity with cognitive and neuropsychological standardised tests.
1 QS4-G shows good predictive validity for cognitive development and neuropsychological abilities. 2 It provides area-specific cut-off. 3 It can be a useful screening tool in preventive medicine as well as research.
The importance of surveillance and screening for developmental problems in primary care has been well recognised. It is particularly important in follow-up programmes and outcome studies for children already known to be at risk for neuropsychological disorders (e.g. children with perinatal problems in particular preterms; children with a genetic, endocrinological, cardiac or neurological pathology as well as any Correspondence: Dr Anna Maria Dall’Oglio, Unit of Clinical Psychology, Department of Neuroscience and Neurorehabilitation, Bambino Gesù Children’s Hospital – IRCCS, Viale Ferdinando Baldelli 48, 00154 Rome, Italy. Fax: 0039 06 68592450; email: [email protected] Conflict of interest: None declared. Accepted for publication 24 October 2014.
600
chronic disease). The primary goal of large-scale screening programmes is to identify children in need of further evaluations as well as to describe typical neuropsychological profiles of specific clinic populations. Moreover, screening tools are often used in general and developmental paediatrics, as well as in schools, for preventive care programmes. Evaluation through questionnaires and interviews is often used for these goals.1,2 When considering child development, a primary concern in selecting questionnaires is predictive validity, which refers to the tool’s ability to predict accurately subjects who are at risk for future problems versus those who are not. Predictive validity requires longitudinal analysis through the use of follow-up studies that compare screening test scores with later performance abilities of the same subjects.
Journal of Paediatrics and Child Health 51 (2015) 600–607 © 2014 The Authors Journal of Paediatrics and Child Health © 2014 Paediatrics and Child Health Division (Royal Australasian College of Physicians)
F Bevilacqua et al.
In a previous study, we constructed and validated a parental questionnaire (QS4-G: Parent Questionnaire for the Developmental Evaluation of 4-Year-Old) designed to evaluate neuropsychological and behavioural developmental status of 4-year-old children and to identify children at risk for whom further evaluation will be necessary.2 The QS4-G consists of 93 questions (items): 87 are multiple-choice (‘often’, ‘sometimes’, ‘never’) and 6 have two possible answers (‘yes’, ‘no’). Its completion requires an average of 15 min and its scoring up to 10 min. It was designed to evaluate two main domains (i.e. neuropsychological and behavioural), divided into 10 specific areas. The neuropsychological domain includes items on: language: phonology, vocabulary and morphosyntax, both in production and comprehension (15 items); visual-motor abilities (9); memory/attention (18); motor abilities: gross motor, coordination and balance, fine-motor abilities, praxic abilities and autonomy (13); and lateralisation (3). The behavioural domain includes hyperactivity and social skills (10); stress (11); sleep (6); alimentation (6); and evacuation (2). The validation study showed good reliability, internal consistency (Cronbach’s alpha for specific areas: 0.68–0.79), concurrent validity with cognitive and neuropsychological standardised tests (r = 0.30–0.68) and discriminant validity (sensitivity 0.88, specificity 0.84 for the optimal cut-off of 48.4 predicting a Griffiths Scale Quotient lower than 81). Areaspecific cut-off were also identified through receiver operating characteristic (ROC) curve (area under the curve (AUC) > 0.75 and sensitivity and specificity >0.70). The validation was conducted on preterm and at-term children (263 as a whole).2 The purpose of the present study is to examine the predictive validity of QS4-G in detecting cognitive, neuropsychological and academic difficulties after a period of 4 to 6 years.
Methods This research is a longitudinal prospective study that took place from September 2007 to December 2008 at Bambino Gesù Children’s Hospital in Rome and at a primary school in Rome, Italy. The Bambino Gesù Children’s Hospital is a tertiary referral
Table 1
Predictive validity of QS4-G
institute for hospital care and scientific research that covers more Italian regions and has been accredited by the Joint Commission International in 2006. The primary school involved in the study is a public school of a central middle-class residential area. We enrolled three groups of children representative of the original sample of the validation study of QS4-G questionnaire,2 who reached grades 3 to 5 of primary school. Briefly, the three groups were: (i) healthy preterms (HP), that is, children with gestational age at birth
doi:10.1111/jpc.12784
ORIGINAL ARTICLE
Predictive validity of the Italian parental questionnaire for developmental evaluation at age 4 (QS4-G) Francesca Bevilacqua,1 Patrizia Giannantoni,2 Patrizio Pasqualetti,3,4 Vincenzo M di Ciommo,5 Maria Franca Coletti,1 Lucilla Ravà,5 Maria Cristina Caselli6 and Anna Maria Dall’Oglio1 1
Unit of Clinical Psychology, Department of Neuroscience and Neurorehabilitation, 5Unit of Epidemiology, Bambino Gesù Children’s Hospital – IRCCS, Department of Statistics, Sapienza University of Rome, 3Service of Medical Statistics and Information Technology, Fatebenefratelli Foundation for Health Research and Education, AFaR Division, 4Language and Communication Across Modalities Laboratory (LACAM), Institute of Cognitive Sciences and Technologies (ISTC-CNR) and 6Institute of Cognitive Sciences and Technologies, National Research Council, Rome, Italy 2
Aim: To examine whether the results at 4 years of age of the developmental questionnaire QS4-G can predict the outcome of cognitive, neuropsychological and academic abilities 4–6 years later. The QS4-G is a validated parental questionnaire designed for the screening and surveillance of the neuropsychological and behavioural developmental status of 4-year-olds (93 questions). Methods: Longitudinal prospective study on a subsample of the QS4-G validation original sample was conducted. According to previous results, the sample was divided into two groups: ‘at risk’ and ‘not at risk’. Sensitivity, specificity, accuracy and likelihood ratios were assessed and referred to outcomes. Results: Thirty-five children were classified as ‘not at risk’ and 16 as ‘at risk’. There were significant associations between past QS4-G score and cognitive, neuropsychological and academic abilities 4–6 years later. With the same cut-off identified at the first cross-sectional study, sensitivity and specificity for difficulties in cognitive development were 90% and 83% while in the neuropsychological abilities 62% and 90%, respectively. A lower predictive validity was found for difficulties in academic abilities (sensitivity 43%, specificity 86%). QS4-G specific area scores showed significant correlations with related academic tests at follow-up (rho range: 0.404–0.565, P < 0.005). Conclusions: QS4-G shows good predictive validity for cognitive development and neuropsychological abilities. The risk of false negatives for academic abilities can be reduced by analysing the specific area results of QS4-G, which show good correlations with related tests at follow-up. Key words:
neuropsychological development; parental questionnaire; predictive validity.
What is already known on this topic
What this paper adds
1 QS4-G is a parental questionnaire designed to evaluate neuropsychological and behavioural developmental status of 4-year-old children. 2 QS4-G showed good reliability, internal consistency and discriminant validity. 3 It also showed good concurrent validity with cognitive and neuropsychological standardised tests.
1 QS4-G shows good predictive validity for cognitive development and neuropsychological abilities. 2 It provides area-specific cut-off. 3 It can be a useful screening tool in preventive medicine as well as research.
The importance of surveillance and screening for developmental problems in primary care has been well recognised. It is particularly important in follow-up programmes and outcome studies for children already known to be at risk for neuropsychological disorders (e.g. children with perinatal problems in particular preterms; children with a genetic, endocrinological, cardiac or neurological pathology as well as any Correspondence: Dr Anna Maria Dall’Oglio, Unit of Clinical Psychology, Department of Neuroscience and Neurorehabilitation, Bambino Gesù Children’s Hospital – IRCCS, Viale Ferdinando Baldelli 48, 00154 Rome, Italy. Fax: 0039 06 68592450; email: [email protected] Conflict of interest: None declared. Accepted for publication 24 October 2014.
600
chronic disease). The primary goal of large-scale screening programmes is to identify children in need of further evaluations as well as to describe typical neuropsychological profiles of specific clinic populations. Moreover, screening tools are often used in general and developmental paediatrics, as well as in schools, for preventive care programmes. Evaluation through questionnaires and interviews is often used for these goals.1,2 When considering child development, a primary concern in selecting questionnaires is predictive validity, which refers to the tool’s ability to predict accurately subjects who are at risk for future problems versus those who are not. Predictive validity requires longitudinal analysis through the use of follow-up studies that compare screening test scores with later performance abilities of the same subjects.
Journal of Paediatrics and Child Health 51 (2015) 600–607 © 2014 The Authors Journal of Paediatrics and Child Health © 2014 Paediatrics and Child Health Division (Royal Australasian College of Physicians)
F Bevilacqua et al.
In a previous study, we constructed and validated a parental questionnaire (QS4-G: Parent Questionnaire for the Developmental Evaluation of 4-Year-Old) designed to evaluate neuropsychological and behavioural developmental status of 4-year-old children and to identify children at risk for whom further evaluation will be necessary.2 The QS4-G consists of 93 questions (items): 87 are multiple-choice (‘often’, ‘sometimes’, ‘never’) and 6 have two possible answers (‘yes’, ‘no’). Its completion requires an average of 15 min and its scoring up to 10 min. It was designed to evaluate two main domains (i.e. neuropsychological and behavioural), divided into 10 specific areas. The neuropsychological domain includes items on: language: phonology, vocabulary and morphosyntax, both in production and comprehension (15 items); visual-motor abilities (9); memory/attention (18); motor abilities: gross motor, coordination and balance, fine-motor abilities, praxic abilities and autonomy (13); and lateralisation (3). The behavioural domain includes hyperactivity and social skills (10); stress (11); sleep (6); alimentation (6); and evacuation (2). The validation study showed good reliability, internal consistency (Cronbach’s alpha for specific areas: 0.68–0.79), concurrent validity with cognitive and neuropsychological standardised tests (r = 0.30–0.68) and discriminant validity (sensitivity 0.88, specificity 0.84 for the optimal cut-off of 48.4 predicting a Griffiths Scale Quotient lower than 81). Areaspecific cut-off were also identified through receiver operating characteristic (ROC) curve (area under the curve (AUC) > 0.75 and sensitivity and specificity >0.70). The validation was conducted on preterm and at-term children (263 as a whole).2 The purpose of the present study is to examine the predictive validity of QS4-G in detecting cognitive, neuropsychological and academic difficulties after a period of 4 to 6 years.
Methods This research is a longitudinal prospective study that took place from September 2007 to December 2008 at Bambino Gesù Children’s Hospital in Rome and at a primary school in Rome, Italy. The Bambino Gesù Children’s Hospital is a tertiary referral
Table 1
Predictive validity of QS4-G
institute for hospital care and scientific research that covers more Italian regions and has been accredited by the Joint Commission International in 2006. The primary school involved in the study is a public school of a central middle-class residential area. We enrolled three groups of children representative of the original sample of the validation study of QS4-G questionnaire,2 who reached grades 3 to 5 of primary school. Briefly, the three groups were: (i) healthy preterms (HP), that is, children with gestational age at birth
