Opinion
VIEWPOINT
David M. Rubin, MD, MSCE PolicyLab at the Children's Hospital of Philadelphia, Philadelphia, Pennsylvania, and Department of Pediatrics, Perelman School of Medicine at the University of Pennsylvania, Philadelphia. Meredith L. Curtis, BA PolicyLab at the Children's Hospital of Philadelphia, Philadelphia, Pennsylvania, and Perelman School of Medicine at the University of Pennsylvania, Philadelphia. Meredith Matone, MHS PolicyLab at the Children's Hospital of Philadelphia, Philadelphia, Pennsylvania.
Corresponding Author: David M. Rubin, MD, MSCE, PolicyLab at the Children's Hospital of Philadelphia, Department of Pediatrics, Perelman School of Medicine at the University of Pennsylvania, 34th St and Civic Center Blvd, CHOP N 3535, Ste 1533, Philadelphia, PA 19104 ([email protected] .edu).
Child Abuse Prevention and Child Home Visitation Making Sure We Get It Right Home visitation to at-risk families is an important strategy for improving early-childhood outcomes. On the heels of numerous trials demonstrating many positive and sustained outcomes for mothers and children, the Affordable Care Act in 2010 allocated $1.5 billion to this initiative and recently, the White House proposed an appropriation of $15 billion over 10 years to strengthen and expand programs across the United States. This substantial public investment in home visiting is a tremendous achievement for researchers who have focused on early childhood as a crucial time period to pursue strategies that alter lifetime trajectories of at-risk children. As these programs rapidly disseminate across the country, researchers must address issues and challenges that may arise and strengthen programs as needed. To this point, child abuse prevention strikes a cautionary note. Prevention of child maltreatment has been a significantpositionofadvocacyforhome-visitingprograms.This outcome is frequently cited as one of the service’s strengths and moreover, has been included in estimates of program cost-effectiveness.1,2 However, a deeper review of the trial evidence is inconclusive. First, the lack of standard outcomes has made comparison of outcomes between program models difficult. Studies vary from more proximal outcomes of harsh parenting practice to health care encounters for injury or official child protective service reports. When comparisons have been made, the results are mixed. The most recent 2013 US Preventive Services Task Force systematic review of homevisitation programs found that when compared with earlier trials, 2 programs (Child First and the Nurse Family Partnership) either reduced reports of abuse or of death due to all causes over a long follow-up. The review also included 4 other high-quality randomized controlled trials that did not find effects on reported child maltreatment.1 AnearlierreviewbyReynoldsetal3 characterizedtheoverall evidence base as relatively weak because of a lack of shared benefit across program models. Second, interpreting trial data has also been challenging due to limitations in study design and risk of bias. Many trials lacked power to detect the rare outcome of abuse over reasonable follow-up durations. Furthermore, many investigators suggest that surveillance bias among visited families might explain the lack of effect observed in some studies. Yet research on the effect of surveillance on child abuse reporting by parent support programs found small influence during the period families were actively enrolled, no excess bias attributed to home visitation, and no evidence that homevisited families were reported for less-severe maltreatment.4 Additionally, many home-visitation programs struggle to retain families once the child is born, weakening this argument. Researchers must therefore
be careful to write off inconclusive data as solely the product of a surveillance bias, masking the program’s true benefit—such is a tenuous position. Uncertainty around the benefit of home-visiting services on child maltreatment reduction is a strong argument for the continued investment in postimplementation evaluation. This is also a time of great opportunity because the implementation environment will expose much larger groups of families to home-visiting services, allowing for robust analysis on a rare outcome such as child maltreatment, and obviating the concerns about power from the original trials. At this time, evaluators should understand that the challenges of expanding programs will be great because implementation across diverse social service agencies and into geographic areas and populations not well-studied in smaller trials may be problematic. Many programs will operate without the resources of clinical trials and face local barriers to implementation and program fidelity.5 Such concerns make it even more important for programs to sustain their commitment to successful implementation and continuing robust evaluations. Although postimplementation data are beginning to emerge, 2 recent studies suggest home-visitation programs may be encountering some challenges. The first, a quasi-experimental evaluation of Nurse Family Partnership in Pennsylvania following the program’s statewide dissemination, found that over a 6-year period, 5016 nurse-visited families were no less likely than 16 704 matched nonvisited families to have serious head injuries or other injuries suspicious for abuse.6 After statewide implementation of Healthy Families Massachusetts (a program using the Healthy Families America curriculum with paraprofessional home visitors), a randomized controlled trial conducted among 840 women at 18 sites found mothers in the home-visiting group were equally, if not more likely, to have a substantiated report of child maltreatment than those in the nonvisited group.7 Certainly researchers should be cautious about the limitations of these postimplementation studies, but shouldnotcompletelydiscountthem.Althoughonestudy was a randomized trial,7 the other was an observational study.6 The observational design might introduce selection bias in the absence of randomization, and even the best attempts at controlling for confounding can be incomplete. However, observational studies include much larger samples and report real-world outcomes and distribution of resources. Generalizability is also a concern because the programs studied in both evaluations represented 1 model in a single state and therefore might not be representative of the performance of other program models or even the same program models implemented
jamapediatrics.com
JAMA Pediatrics January 2014 Volume 168, Number 1
Copyright 2014 American Medical Association. All rights reserved.
Downloaded From: http://archpedi.jamanetwork.com/ by a University of Georgia User on 05/30/2015
5
Opinion Viewpoint
in other geographic areas. But both studies evaluated 2 highly credible programs using widely accepted postimplementation methods, and as noted previously, join a primary literature that is quite inconclusive on child abuse prevention. Regardless of how one interprets the results of clinical trials and postimplementation studies, researchers and advocates should heed the warnings from inconclusive data. In public advocacy for programs, clinicians must be careful in overreaching on child abuse prevention at the cost of undermining the many terrific benefits that home visiting does provide for high-risk families. We have visited many of the programs in Pennsylvania and can attest that there are both measured and unmeasured ways in which these programs help families. Our limited understanding of why we have not achieved consistent success in child abuse prevention must be acknowledged. Is it the engagement and retention of families in programs? Are there local barriers in organizational performance or community service integration that might be harming program fidelity? Researchers may
Washington. Olympia, Washington: Washington State Institute for Public Policy; 2008. Document No. 08-07-3901.
ARTICLE INFORMATION Published Online: November 11, 2013. doi:10.1001/jamapediatrics.2013.3865. Conflict of Interest Disclosures: None reported. REFERENCES 1. Selph SS, Bougatsos C, Blazina I, Nelson HD. Behavioral interventions and counseling to prevent child abuse and neglect: a systematic review to update the US Preventive services task force recommendation. Ann Intern Med. 2013;158(3):179-190. 2. Lee S, Aos S, Miller M. Evidence-based programs to prevent children from entering and remaining in the child welfare system: benefits and costs for
6
yet find that home-visiting programs are only as strong as the community fabric of coordinated programs in which they are inserted. Distinguishing the root causes through implementation research will be essential. Investing time in understanding these barriers may also prevent the erosion of other well-established benefits of home visitation. Researchers cannot forget that although programs of home visitation were originally disseminated because of their broad benefits in maternal child health, they also showed promise in addressing the critical public health problem of preventing injury to vulnerable children. Home visiting researchers who are proactive about acknowledging and addressing the increasingly inconclusive data on this outcome will be 1 step ahead of those who would seek to interpret these data as indicative of more general concerns with program expansion. If such a review helps steward a conversation of better engaging families and weaving together home-visitation programs across a continuum of early childhood services, then the dialogue we have created will not have been in vain.
3. Reynolds AJ, Mathieson LC, Topitzes JW. Do early childhood interventions prevent child maltreatment? a review of research. Child Maltreat. 2009;14(2):182-206. 4. Chaffin M, Bard D. Impact of intervention surveillance bias on analyses of child welfare report outcomes. Child Maltreat. 2006;11(4):301-312. 5. Fixsen D, Blase K, Metz A, Van Dyke M. Statewide implementation of evidence-based programs. Except Child. 2013;79(2):213-230. http://cec.metapress.com/content /J47T21524330Q807. Accessed October 4, 2013.
6. Matone M, O’Reilly ALR, Luan X, Localio AR, Rubin DM. Emergency department visits and hospitalizations for injuries among infants and children following statewide implementation of a home visitation model. Matern Child Health J. 2012;16(9):1754-1761. 7. Easterbrooks MA, Jacobs FH, Bartlett JD, et al. Initial findings from a randomized, controlled trial of Healthy Families Massachusetts: early program impacts on young mothers’ parenting. Pew states home visitation evaluation. http://www.pewstates .org/uploadedFiles/PCS_Assets/2013/Healthy _Families_Massachusetts_executive_summary.pdf. Accessed April 4, 2013.
JAMA Pediatrics January 2014 Volume 168, Number 1
Copyright 2014 American Medical Association. All rights reserved.
Downloaded From: http://archpedi.jamanetwork.com/ by a University of Georgia User on 05/30/2015
jamapediatrics.com
VIEWPOINT
David M. Rubin, MD, MSCE PolicyLab at the Children's Hospital of Philadelphia, Philadelphia, Pennsylvania, and Department of Pediatrics, Perelman School of Medicine at the University of Pennsylvania, Philadelphia. Meredith L. Curtis, BA PolicyLab at the Children's Hospital of Philadelphia, Philadelphia, Pennsylvania, and Perelman School of Medicine at the University of Pennsylvania, Philadelphia. Meredith Matone, MHS PolicyLab at the Children's Hospital of Philadelphia, Philadelphia, Pennsylvania.
Corresponding Author: David M. Rubin, MD, MSCE, PolicyLab at the Children's Hospital of Philadelphia, Department of Pediatrics, Perelman School of Medicine at the University of Pennsylvania, 34th St and Civic Center Blvd, CHOP N 3535, Ste 1533, Philadelphia, PA 19104 ([email protected] .edu).
Child Abuse Prevention and Child Home Visitation Making Sure We Get It Right Home visitation to at-risk families is an important strategy for improving early-childhood outcomes. On the heels of numerous trials demonstrating many positive and sustained outcomes for mothers and children, the Affordable Care Act in 2010 allocated $1.5 billion to this initiative and recently, the White House proposed an appropriation of $15 billion over 10 years to strengthen and expand programs across the United States. This substantial public investment in home visiting is a tremendous achievement for researchers who have focused on early childhood as a crucial time period to pursue strategies that alter lifetime trajectories of at-risk children. As these programs rapidly disseminate across the country, researchers must address issues and challenges that may arise and strengthen programs as needed. To this point, child abuse prevention strikes a cautionary note. Prevention of child maltreatment has been a significantpositionofadvocacyforhome-visitingprograms.This outcome is frequently cited as one of the service’s strengths and moreover, has been included in estimates of program cost-effectiveness.1,2 However, a deeper review of the trial evidence is inconclusive. First, the lack of standard outcomes has made comparison of outcomes between program models difficult. Studies vary from more proximal outcomes of harsh parenting practice to health care encounters for injury or official child protective service reports. When comparisons have been made, the results are mixed. The most recent 2013 US Preventive Services Task Force systematic review of homevisitation programs found that when compared with earlier trials, 2 programs (Child First and the Nurse Family Partnership) either reduced reports of abuse or of death due to all causes over a long follow-up. The review also included 4 other high-quality randomized controlled trials that did not find effects on reported child maltreatment.1 AnearlierreviewbyReynoldsetal3 characterizedtheoverall evidence base as relatively weak because of a lack of shared benefit across program models. Second, interpreting trial data has also been challenging due to limitations in study design and risk of bias. Many trials lacked power to detect the rare outcome of abuse over reasonable follow-up durations. Furthermore, many investigators suggest that surveillance bias among visited families might explain the lack of effect observed in some studies. Yet research on the effect of surveillance on child abuse reporting by parent support programs found small influence during the period families were actively enrolled, no excess bias attributed to home visitation, and no evidence that homevisited families were reported for less-severe maltreatment.4 Additionally, many home-visitation programs struggle to retain families once the child is born, weakening this argument. Researchers must therefore
be careful to write off inconclusive data as solely the product of a surveillance bias, masking the program’s true benefit—such is a tenuous position. Uncertainty around the benefit of home-visiting services on child maltreatment reduction is a strong argument for the continued investment in postimplementation evaluation. This is also a time of great opportunity because the implementation environment will expose much larger groups of families to home-visiting services, allowing for robust analysis on a rare outcome such as child maltreatment, and obviating the concerns about power from the original trials. At this time, evaluators should understand that the challenges of expanding programs will be great because implementation across diverse social service agencies and into geographic areas and populations not well-studied in smaller trials may be problematic. Many programs will operate without the resources of clinical trials and face local barriers to implementation and program fidelity.5 Such concerns make it even more important for programs to sustain their commitment to successful implementation and continuing robust evaluations. Although postimplementation data are beginning to emerge, 2 recent studies suggest home-visitation programs may be encountering some challenges. The first, a quasi-experimental evaluation of Nurse Family Partnership in Pennsylvania following the program’s statewide dissemination, found that over a 6-year period, 5016 nurse-visited families were no less likely than 16 704 matched nonvisited families to have serious head injuries or other injuries suspicious for abuse.6 After statewide implementation of Healthy Families Massachusetts (a program using the Healthy Families America curriculum with paraprofessional home visitors), a randomized controlled trial conducted among 840 women at 18 sites found mothers in the home-visiting group were equally, if not more likely, to have a substantiated report of child maltreatment than those in the nonvisited group.7 Certainly researchers should be cautious about the limitations of these postimplementation studies, but shouldnotcompletelydiscountthem.Althoughonestudy was a randomized trial,7 the other was an observational study.6 The observational design might introduce selection bias in the absence of randomization, and even the best attempts at controlling for confounding can be incomplete. However, observational studies include much larger samples and report real-world outcomes and distribution of resources. Generalizability is also a concern because the programs studied in both evaluations represented 1 model in a single state and therefore might not be representative of the performance of other program models or even the same program models implemented
jamapediatrics.com
JAMA Pediatrics January 2014 Volume 168, Number 1
Copyright 2014 American Medical Association. All rights reserved.
Downloaded From: http://archpedi.jamanetwork.com/ by a University of Georgia User on 05/30/2015
5
Opinion Viewpoint
in other geographic areas. But both studies evaluated 2 highly credible programs using widely accepted postimplementation methods, and as noted previously, join a primary literature that is quite inconclusive on child abuse prevention. Regardless of how one interprets the results of clinical trials and postimplementation studies, researchers and advocates should heed the warnings from inconclusive data. In public advocacy for programs, clinicians must be careful in overreaching on child abuse prevention at the cost of undermining the many terrific benefits that home visiting does provide for high-risk families. We have visited many of the programs in Pennsylvania and can attest that there are both measured and unmeasured ways in which these programs help families. Our limited understanding of why we have not achieved consistent success in child abuse prevention must be acknowledged. Is it the engagement and retention of families in programs? Are there local barriers in organizational performance or community service integration that might be harming program fidelity? Researchers may
Washington. Olympia, Washington: Washington State Institute for Public Policy; 2008. Document No. 08-07-3901.
ARTICLE INFORMATION Published Online: November 11, 2013. doi:10.1001/jamapediatrics.2013.3865. Conflict of Interest Disclosures: None reported. REFERENCES 1. Selph SS, Bougatsos C, Blazina I, Nelson HD. Behavioral interventions and counseling to prevent child abuse and neglect: a systematic review to update the US Preventive services task force recommendation. Ann Intern Med. 2013;158(3):179-190. 2. Lee S, Aos S, Miller M. Evidence-based programs to prevent children from entering and remaining in the child welfare system: benefits and costs for
6
yet find that home-visiting programs are only as strong as the community fabric of coordinated programs in which they are inserted. Distinguishing the root causes through implementation research will be essential. Investing time in understanding these barriers may also prevent the erosion of other well-established benefits of home visitation. Researchers cannot forget that although programs of home visitation were originally disseminated because of their broad benefits in maternal child health, they also showed promise in addressing the critical public health problem of preventing injury to vulnerable children. Home visiting researchers who are proactive about acknowledging and addressing the increasingly inconclusive data on this outcome will be 1 step ahead of those who would seek to interpret these data as indicative of more general concerns with program expansion. If such a review helps steward a conversation of better engaging families and weaving together home-visitation programs across a continuum of early childhood services, then the dialogue we have created will not have been in vain.
3. Reynolds AJ, Mathieson LC, Topitzes JW. Do early childhood interventions prevent child maltreatment? a review of research. Child Maltreat. 2009;14(2):182-206. 4. Chaffin M, Bard D. Impact of intervention surveillance bias on analyses of child welfare report outcomes. Child Maltreat. 2006;11(4):301-312. 5. Fixsen D, Blase K, Metz A, Van Dyke M. Statewide implementation of evidence-based programs. Except Child. 2013;79(2):213-230. http://cec.metapress.com/content /J47T21524330Q807. Accessed October 4, 2013.
6. Matone M, O’Reilly ALR, Luan X, Localio AR, Rubin DM. Emergency department visits and hospitalizations for injuries among infants and children following statewide implementation of a home visitation model. Matern Child Health J. 2012;16(9):1754-1761. 7. Easterbrooks MA, Jacobs FH, Bartlett JD, et al. Initial findings from a randomized, controlled trial of Healthy Families Massachusetts: early program impacts on young mothers’ parenting. Pew states home visitation evaluation. http://www.pewstates .org/uploadedFiles/PCS_Assets/2013/Healthy _Families_Massachusetts_executive_summary.pdf. Accessed April 4, 2013.
JAMA Pediatrics January 2014 Volume 168, Number 1
Copyright 2014 American Medical Association. All rights reserved.
Downloaded From: http://archpedi.jamanetwork.com/ by a University of Georgia User on 05/30/2015
jamapediatrics.com
