583
ATTRIBUTES OF SUCCESSFUL PROGRAMS FOR ADOLESCENTS: WHAT WORKS* MICHAEL A. BAILIN, J.D., M.U.S. President Public/Private Ventures Philadelphia, Pennsylvania
IADDRESS THIS ISSUE from a very practical vantage point. Situated at Public/Private Ventures, a not-for-profit research and development organization funded by national and local foundations, government agencies, and corporations, we develop, test and assess programs for at-risk young people; in other words, we try to find out "what works." Our perspective is that of an organization that defines adolescence as spanning all the teen-age years; that subsumes under "at risk" the whole range of economic, educational, and social barriers to responsible, productive adulthood; that construes the word "program" to encompass all its elements, including its funding, scale, targeting strategy, delivery mechanisms, and practical utility, as well as the shape of the model itself; whose activities relate primarily to urban youth who are economically, socially, and educationally disadvantaged; and whose ultimate goal is to contribute to the development of public social policy and better programming in public institutions. From that perspective, I stress from the outset that no conclusive evidence supports either the pessimistic or the optimistic view of public program efficacy that divides the policy community today. The pessimistic view was crystallized in Charles Murray's 1984 book, Losing Ground, 1 which argues that social programs have done little good for poor people, and that in fact they may exacerbate the very problems they set out to tackle. He indicts a wide range of programs and suggests that the poor might be better served by what George Gilder calls "the spur of poverty. " In Murray's analysis, interventions such as those devised during the War on Poverty and its aftermath are part of the problem. Indeed, he argues, they are the disease, so funding them merely undermines our impulse to help the disadvantaged. By the end of the 1980s, Murray's characterization was being countered not only by critiques of his evidence, but by a round of books and reports *Presented as part of a Conference on Strengths and Potentials of Adolescence held by the Committee on Public Health of the New York Academy of Medicine March 8, 1991.
Vol. 67, No. 6, November-December 1991
584
M.A. BAILIN
presenting their evidence of the effectiveness of social programs, particularly those for children and adolescents in poverty. These new distillations began to offer us a glimpse of programs and principles that appear to succeed. The most famous of these volumes is Lisbeth Schorr's Within Our Reach.2 However, the new wave also includes Joy Dryfoos' Adolescents At-Risk,3 the American Psychological Association's 14 Ounces of Prevention,4 the National League of Cities' Children, Families, and Cities,5 publications of the Committee on Economic Development and the Children's Defense Fund, the W.T. Grant Foundation Commission's The Forgotten Half,6 and many, many others. All of them mine two decades of poverty fighting in an effort to discover what it is that does work for poor children, and they reach some common conclusions. First, they conclude that some programs do indeed seem to work. Within Our Reach describes 17 in some detail from the fields of family support, child welfare, and education. Included in the list are such nationally acclaimed efforts as Head Start and I Have a Dream, and less well-known ones such as Homebuilders in Tacoma, Washington. Adolescents At-Risk surveyed a much wider group of nearly 100 programs judged to be successful in the fields of delinquency, substance abuse, teen pregnancy, and educational remediation. Second, these researchers all tend to see common elements among effective programs. They are comprehensive, intensive, and flexible; they provide youth a great deal of individualized attention; they deal with the child in the context of family and community; they are characterized by strong management; they take a long and developmental view of providing help. The third common feature I see in these reports has more to do with tone than substance. They are optimistic. In her introduction to Within Our Reach Judith Viorst rejoices that we now have a blueprint to overcome what she calls a sense of "helplessness and hopelessness" about rescuing adolescents in poverty. In this spirit of optimism, Schorr contends that a "persuasive body of research and experience shows that the knowledge exists on which to build the interventions that will significantly improve outcomes among children and youth at-risk, including the children who are Truly Disadvantaged." Dryfoos agrees, declaring that "enough is known to greatly improve the potential life course for high-risk children in the United States." As president of an organization whose mandate is to locate and to test effective strategies for disadvantaged youth, I am naturally sympathetic to the considered opinion that concludes that such strategies are worthwhile, that Bull. N.Y. Acad. Med.
UWHAT WORKS WORKS
WHAT
585
they provide the lessons required for action, and that we are in a position to act decisively. However, I am concerned about the current tide of optimism that is attempting to replace the earlier bout of pessimism. While I have no qualms about the common-sense truth that some programs help high-risk adolescents and that they share some general features, I am as troubled by the assertion that we have our hands around some "cure" as I am by the earlier notion that social intervention is spreading the "illness." Unlike medicine, social policy research is at an incipient stage of development. We have little solid theory to rest upon, and our methods are primitive. We still have a hard time agreeing about the problem. One observer suggests, borrowing from medicine's history, that we have only recently started washing our instruments. Even the findings from our greatest success stories are much thinner than they might first appear. My reading of the research literature suggests that even for many of the best studies there is more statistical significance than actual significance or change in people's lives. Much of our confidence in Head Start rests on data from the Perry preschool experience involving just a handful of children; Job Corps' often-invoked success is based in large measure on crime that was not committed by young people while they were in the program; attrition in many of our most heralded efforts hovers near 50%; most successful programs touted in the reports I mentioned are small and idiosyncratic, and are yet to be seriously evaluated. To my mind, and this is supported by Public/Private Ventures' investigations, relatively few programs have been shown to work, even relatively few so-called exemplary programs work, and all too often those that do work at all produce only modest aggregate effects and uneven subgroup effects for the variety of participants who enroll. My concern about the state of our knowledge about what works is heightened because developing knowledge is only one fraction of the battle. The other fractions are even larger, pose greater challenges, and, until these are addressed squarely, bits of evidence about particular programs will neither inform the whole campaign nor provide the foundation on which we can move forward to assist large numbers of at-risk adolescents gain productive adulthood. THE LARGE CHALLENGES
In our work at Public/Private Ventures, three major challenges emerge from examination of "Why don't we know better about what works and what doesn't?" There is a clue in the two positions I referred to a moment ago: Vol. 67, No. 6, November-December 1991
586
586
BARN M.A. BAILIN
pessimists say that large public programs are not powerful enough to produce positive aggregate results and may even hurt people; optimists say that small, idiosyncratic programs do help some people; yet neither really offers convincing evidence to support their positions. I suggest that this is so because, in the public policy arena, we pursue program development and research without even a crude theory about what youth need to be successful; we test individual program components that address discrete fragments of a youth's problems rather than the whole youngster with all his needs; and we fail to establish convincing standards of success or how to transfer successful programs throughout the field. Therefore, I think we face three challenges. DEVELOP THEORETICAL FOUNDATIONS
First, we need to develop that crude theory of what it takes for a youth to have a reasonable chance of being successful -crude "because policy development, in the final analysis, is essentially a political process, and "reasonable chance" because even our best efforts are likely only to convert crises to problems, not to eliminate them entirely. My conviction about need for a theoretical foundation for our efforts emerges from Public/Private Ventures' experience and that of others working with youth. During the late 1970s and early 1980s, programs were developed and tested that addressed discrete pieces of a young person's experiencelack of work experience, lack of basic academic skill, lack of occupational skill, poor work attitudes, teen-age parenting, antisocial behavior-one at a time, and with little recognition of the variety of youth needs based on their age and stage of development. We should not have been surprised that we found few long-term effects. Gradually, we came to recognize the need for-and to require in our own work-a conceptual justification of what we meant by successful development based on theory or evidence for all the employment programs we support or evaluate. This may slow the pace of our work, to be sure; but quick fixes yield programs like the Family Support Act and the Job Training Partnership Act that have no theoretical or conceptual basis, even though we judge them as though they had. For us, a theory of what programs need to do to help at-risk youth become successful adults is beginning to emerge: They need to provide real opportunities -appropriate jobs and/or training experiences; they need to provide access to those opportunites by proffering everything from guided referrals to transportation to the suburbs; they need to Bull. N.Y. Acad. Med.
WHAT WORKS WORKS
WHAT
587
teach the personal and technical skills necessary for youth to take advantage of the available opportunities -and we now have a pretty good handle on what those skills are; and they need to assure participating youth the basic conditions of health, safety, adult caring and guidance so essential to develop those skills and to enable them to take advantage of opportunities. Indeed, there are exceptional youth who do not need all four components and exceptional programs that get results without providing them all. But these are typically boutique projects that have not demonstrated that they can be brought to scale, and good public policy must aim at addressing the needs of significant numbers of people. At first glance, these four components seem obvious enough and feasible to implement in programs. In fact, they are difficult to achieve in practice: social and economic trends push them beyond the reach of many urban at-risk youth. The number and quality of job opportunities are fewer; ways to access them are less available; required skill levels are rising; and the basic conditions of life- according to unanimous reports from the field -are debilitating and getting worse all the time. The point, in sum, is that a firm theoretical foundation would drive us to address all four aspects of youth's needs in an organized, continuous youthserving system. Today there is no system, and most programs focus only on one, or at most two, of the basic requirements. That is why their results are often so weak. Early interventions may provide basic skills, but usually do not deal with life conditions and seldom reduce the need for later assistance. Later interventions are therefore overmatched and asked to cover lost ground too late in the game. IMPROVE RESEARCH TARGETS AND PROCEDURES
Our second challenge is to develop a social research process that maximizes our chances of finding results -one that tests theories, not programs and one that tests them properly. As I have argued, testing programs that have just one or two components lead us nowhere. If youth need a range of services, testing programs that offer only a few and holding them to standards of long-term, life-changing success only dooms even the most promising to failure. Hence research must, from the outset, focus on program ideas with strong chances for success. In our testing procedures we are clearly still in adolescence ourselves. Funding often rides a policy wave or fad, so a research demonstration must start up quickly and get results fast. Medical research has long recognized the folly of such a procedure. There, knowledge development is pursued through Vol. 67, No. 6, November-December 1991
588
MA. M.A. BAILIN
repeated trials and modifications before formal testing begins, and very longterm follow-up is the rule. For social policy research, preliminaries and the long view are unheard-of luxuries. Little wonder we do not get strong results from our premature, short-term tests. And as for the government's role in developing and testing effective youth strategies, everything now militates against success. First, any demonstration responding to the four goals posed by my working theory-opportunities, access, skills, and life support-would require collaboration by at least several federal agencies, at a minimum, the Departments of Labor, Health and Human Services and Education. But, as we all know, funding streams do not flow together, mandates seldom converge, categories are either exclusionary and overlapping, and spoken languages are unique, to say the least. Second, a four-pronged approach requires, at the local delivery level, flexibility in responding to local conditions and resources and discretion in targeting and delivering services. But, as we know, that is not the norm. We now have traditionally trained professionals and conventional approaches that prize accountability for the expenditure of public funds. Third, demonstrating our theory would require delivery of intense, individualized services. But the reality is that public programs are required to serve the most people possible and to ensure equity, both in the face of falling dollar amounts. Finally, testing a conceptually sound system would involve both preventive and long-term services, while public pressure mounts for persuasive short-term payoffs, and funding of any service is unpredictable. TRANSFER RESEARCH-BASED KNOWLEDGE INTO THE FIELD The third challenge is to find ways to implement broadly programs that do demonstrate success. We simply do not have and desperately need public and bureaucratic support and structure that includes incentives and resources to encourage the transfer of effective programs and practices. Even when the demonstration/testing process does produce good results, our public infrastructure does not know what to do with them, beyond mailing out reports and holding occasional workshops. This produces little in the way of improved programming. It may produce better substantive policies since Congressional staffers do, more and more, read these reports. But in our experience, even that seldom improves programs and services for at-risk youth. There are simply no incentives or resources to ensure that effective programming is transferred and replicated. Our system puts too much focus on local invention and ownership rather than on effectiveness. And, of course, Bull. N.Y. Acad. Med.
WHAT WORKS WORKS
WHAT
589
there remains the issue of field capacity- staff terribly overburdened, insufficiently trained, and not very sophisticated in the replication and transferring of knowledge. The example of our Summit Training and Education Program (STEP) program is illustrative. It is a well-researched model with strong short-term impacts serving an underserved group of low-income young teen-agers with educational deficits that local service deliverers were required by federal legislation to address. It offers localities a tested, packaged, relatively cheap way to comply. But no mechanism in place enables localities to seize such opportunities. It took a specially planned effort by Public/Private Ventures, an intermediary, to market the model, to get the various actors together to provide local support, to train the staff and teachers, to provide the curricula, to show the localities how to run the program, monitor operations, and begin to construct a national network of STEP programs. Institutionalizing this replication function is clearly necessary. We may be seeing some movement in this direction in the fields of employment and training. Senator Simon introduced an amendment to the relevant federal law, the Job Training Partnership Act, last year that would have set aside a fund for replication. And the Departments of Labor and Health and Human Services have shown increasing concern about staff training in the use of model programs. But we still proceed as if good programs can always be a product of local good will and intention. Our experience shows that is not the case. Local people have neither the time nor the resources to develop new curricula, to articulate new approaches to learning, to rationalize and to document new ways to break down and to analyze jobs. Resources to do these things should lie at the national level, as should the resources to test them and to teach them. Our federal structure needs to see the proliferation of effective programs as one of its major responsibilities. Before moving on, I want to leaven the discouraging news just reviewed by noting that we see progress in meeting all three challenges. Funders, policymakers, and social observers increasingly recognize the necessity to construct and to test basic theories about what works, to support sensible interdepartmental demonstrations of approaches with solid conceptual underpinnings, and to develop a science of implementation that includes both structures and incentives. And though public programming and policy do not develop as rationally as we might wish, we see the legislative process paying increasing attention to evidence. Buttressing that trend is very important; it is our best hope. Without Vol. 67, No. 6, November-December 1991
590
590
BARN MA. BAELIN M.A.
evidence, and sometimes even with it, the same facts are open to different interpretations and conclusions by people governed by different political convictions, different public agendas. We need something more solid to go on. WHAT WORKS
Despite the deficiencies I have discussed, we have learned some things that can be useful now, both conceptually and practically. At the conceptual, or "broad lesson" level, what Public/Private Ventures is finding reflects consistently what Liz Schorr, Joy Dryfoos, and others have documented. One, we have found that successful interventions for at-risk youth address multiple needs simultaneously, not singly or sequentially. They are, in short, comprehensive and intensive. They overcome fragmentation through staff versatility and flexibility and by active collaboration across bureaucratic and professional boundaries. Two, we have found that successful programs are typically initiated and sustained by exceptional leaders-not because those interventions require exceptional people to implement them, but because our public funding and regulatory environment does so little to encourage, nurture, or sustain such programs. In fact, just the opposite. We need leaders to start and to maintain these programs in spite of the funding and regulatory environment. Three, we have found that these interventions focus as much on fostering a caring, supportive environment as on delivering professional services-not simply for warm-hearted reasons but because many at-risk youth are unreceptive to professional services until they have a sense they are cared for, and trust the adults who staff the invervention. Four, such interventions are usually inventive in the way they deliver services -they do not resemble the usual menu. Five, successful interventions constantly strive to connect youth to the larger society through understanding it, being involved with it. They do not take the safe route of building a program environment that works well so long as the real world does not intrude. They do not build a boundary around the program and, when finished with the youths, dump them into the real world. Let me briefly describe a few real programs we know well that exemplify these principles in practice for different stages of adolescence. For 10- to 14-year-olds there is I Have a Dream (IHAD) the scores of programs that echo Eugene Lang's model, which offers sixth graders a postgraduate tuition guarantee, sponsorship by a powerful financier, and, most critically, constant personal attention by a school-based project coordinator. Bull. N.Y. Acad. Med.
WHAT WORKS
WHAT WORKS
591 591
We spent time last year examining sixth- seventh- and eighth-grade "dreamers," as they are called, in three District of Columbia schools. Starting out skeptics, we finished the first year's look believing that IHAD shows real promise. Feeling special turned out to be an important motivator. The dreamers all spoke of basking in the attention they got from a successful person, as did their parents and their schools. The long-term, consistent, supportive counseling relationship of the project coordinator appears critical. This nonbureaucratic person responds flexibly and individually, with no restrictions on hours of service or what is and is not allowed. The job description says, literally, the project coordinator cares for these youngsters for six years. They need special tutoring; they get it. They need a ride to school; they get it. They need health services, glasses; they get them. IHAD's extracurricular activities involve the youngsters in the larger world and help them to feel a part of it. They go to plays, summer camps, big league ball games, a Japanese restaurant. Their horizons are expanded, they are enriched. After only one year of study, of course, we do not know if the District of Columbia's IHAD's early promise will hold up as youngsters scatter from their middle school to high schools all over the city and as overburdened coordinators possibly burn out. Nor do we know whether public institutions can enlarge or replicate the critical features and potential of this essentially private-sector effort. That is what we shall study and explore over the next years. The second program I shall discuss is for 14- to 15-year-olds. It's Public/Private Venture's STEP, which we designed, piloted, and tested in five cities with nearly 3,000 treatment youths. The model provides work experience and instruction in basic academic and life skills over two summers and continued contact with the program during the intervening school year. For the demonstration, we developed a new way to teach basic and life skills, both of which involve the youths in discussing real life -and introduced them to work at an earlier age than has been traditional. We are still looking at longer-term results, but short-term evidence is impressive: learning losses typically recorded over the summer by disadvantaged youths were eliminated for the treatment group, many of whom actually made gains over the summer in reading and mathematics; STEP youth increased their understanding of what they can do to handle their real life problems; and they came to appreciate and to like real work. But there it ends. Will regular schooling likely build on these gains? Probably not. And will public institutions widely utilize STEP? Not unless they continue to get assistance to do so-as Public/Private Ventures has Vol. 67, No. 6, November-December 1991
592
592
M.A. BAILIN M~~~~~~~~.A. BA..L...
provided to the nearly 100 communities that run STEP. The third program I shall describe is for 18- to 24-year-olds. It is the Center for Employment Training based in California. This is, I think, unique. It minimizes formal education, but integrates exactly what is needed into the practical skills training process. Each trainee selects the occupation he wants to be trained for, is assigned immediately a personal trainer who not only teaches occupational skills but also the academic skills necessary to perform that job -and helps the trainees get the support services they may need to stay in the program. The Center guarantees a job to every completer, a trick made possible by the organization's strong connections with employers, whose jobs they analyze down to every teachable component. The results? Right from the start the young people see real opportunities at the end of training. Early assessment and testing are dispensed with. Trainees get right down to work. They respond to the Center's innovative teaching, which works even with people who have little English. And they are bolstered by a caring environment in which staff members take responsibility to help youth address their life problems. What do these three programs, which deal with young people at different times of their lives, have in common? None started, or survive, dependent on a single and conventional public funding source. Each delivers services in an entirely different way. None insulate youths from the real world; each addressses the connections between the two and guides youths into developing their own. The two that deliver educational and/or technical skills needed substantial resources and time to hone their key techniques, and required more planning and resources to teach these techniques to others. In no instance has replication sprouted locally on its own. In sum, I think we are learning both broad principles and specific practices to help at-risk youth transition to adulthood. But I also think that the broad principles, once understood, are difficult to convert into effective practice at any reasonable scale. That requires an investment of resources to develop, test, refine, and modify-and then build out and transfer to other jurisdictions. Such investments are simply not possible within the current level or structure of public funding. Nor does it seem that we have the necessary political will to address the problems of at-risk youth seriously. THE ULTIMATE CHALLENGE: BUILDING CONCERN
Before concluding, I want to shift to another level of discourse and talk to you more as a worried human being and less as a student of public policy and manager of a research organization. Like most of you, I am worried about these youngsters and that society does not seem to share my concern. Bull. N.Y. Acad. Med.
WHAT WORKS
WHAT WORKS
593
For the first time in our history, children are the poorest age group in the population. And for those children growing up in poverty-in inner-city neighborhoods and housing projects that Nicholas Lemann recently condemned as being "among the worst places in the world to live" -life is full of stress, violence, and misery. Dangers are ever present, supports are few. Parents, themselves under great stress, can help to buffer their children against these circumstances, but it is often impossible for them to do so single-handedly. When they become incapacitated for one reason or another, their children are truly on their own. Missing from our inner-city neighborhoods-and as a matter of fact, as from most of our society-is the informal web of community, the loose collection of kith and kin who assumed responsibility in the past for helping to shape, nurture, develop, and discipline young people. Part of the erosion of this fabric of support and responsibility is attributable to the movement of black middle-class families out of the inner cities, but it is also part of a general weakening in our society of interest in childrenparticularly other people's children. Most parents barely have enough time to talk to their own children, much less their neighbors'. When it comes to children of the poor this indifference is damaging and widespread. Young people growing up poor in inner-city neighborhoods increasingly grow up in an unsupportive world -one in which they are cut off from much informal support in their immediate sphere, while simultaneously being abandoned at a national level by mainstream society. And that is really why people in our business go on these days collecting bits and pieces of research information, common-sense insights, moral intuition, compassion, whatever resources we can muster, all the while examining our progress and trying to redirect our efforts in ways I have sketched. In fact, there really is no viable alternative. Even lacking valid theories or a robust science of implementation, the needs of disadvantaged youth are so deep and pervasive that we must slog on. We do not have the luxury of waiting for a well-designed and fully detailed blueprint or for unassailable research evidence.
CONCLUSION I have called for better conceptual thinking, for better research, and for a science of implementation -but do I really believe that pursuing these goals will move us, as a nation, toward a caring society, one with a coherent family and children's policy, like those that many less rich nations now have? One that invests the resources necessary to ease the terrible problems of poor children and their families? Vol. 67, No. 6, November-December 1991
594
M.A. BAILIN
My own belief has been renewed recently by the rapid growth and popularity of mentoring programs and other initiatives that combine informal adult support, caring, and guidance with education and social services. But I fear that the public and policy interest in these initiatives is founded too much on the mistaken notion that they are cheap and easy. Public/Private Ventures' initial research on these approaches shows that they are, in fact, expensive and difficult to do well. Nevertheless, growing public interest in reconstituting a caring society an interest in part embodied in this wave of mentoring activity for at-risk youth -is encouraging. Its themes -responsibility, reciprocity, participation and interdependence -may be able to bring together diverse positions on the ideological spectrum, and may thereby contribute to generating the political will required to institute the reforms necessary if we are really to help adolescents who are at risk. Moving society to care about its young people may, in the end, be our biggest challenge of all and our greatest accomplishment. REFERENCES 1. Murray, C.A.: Losing Ground-American can Psychological Association, 1988. Social Policy, 1950-1980. New York, 5. National League of Cities; Children, Basic Books, 1984. Families, and Cities. Washington, D.C., 2. Schorr, L. and Schorr, D.: Within Our National League of Cities, 1987. Reach: Breaking the Cycle of Disadvan- 6. Youth and America's Future: The Willtage. New York, Doubleday, 1988. iam T. Grant Foundation Commission on 3. Dryfoos, J.: Adolescents At-Risk: PrevenWork, Family and Citizenship. In: The tion and Prevalence. New York, Oxford Forgotten Half: Pathways to Success For University Press, 1990. America's Youth and Young Families. 4. American Psychological Association; 14 Washington, D.C., William T. Grant Ounces of Prevention: A Casebook for Commission, 1988. Practitioners. Washington, D.C., Ameri-
Bull. N.Y. Acad. Med.
ATTRIBUTES OF SUCCESSFUL PROGRAMS FOR ADOLESCENTS: WHAT WORKS* MICHAEL A. BAILIN, J.D., M.U.S. President Public/Private Ventures Philadelphia, Pennsylvania
IADDRESS THIS ISSUE from a very practical vantage point. Situated at Public/Private Ventures, a not-for-profit research and development organization funded by national and local foundations, government agencies, and corporations, we develop, test and assess programs for at-risk young people; in other words, we try to find out "what works." Our perspective is that of an organization that defines adolescence as spanning all the teen-age years; that subsumes under "at risk" the whole range of economic, educational, and social barriers to responsible, productive adulthood; that construes the word "program" to encompass all its elements, including its funding, scale, targeting strategy, delivery mechanisms, and practical utility, as well as the shape of the model itself; whose activities relate primarily to urban youth who are economically, socially, and educationally disadvantaged; and whose ultimate goal is to contribute to the development of public social policy and better programming in public institutions. From that perspective, I stress from the outset that no conclusive evidence supports either the pessimistic or the optimistic view of public program efficacy that divides the policy community today. The pessimistic view was crystallized in Charles Murray's 1984 book, Losing Ground, 1 which argues that social programs have done little good for poor people, and that in fact they may exacerbate the very problems they set out to tackle. He indicts a wide range of programs and suggests that the poor might be better served by what George Gilder calls "the spur of poverty. " In Murray's analysis, interventions such as those devised during the War on Poverty and its aftermath are part of the problem. Indeed, he argues, they are the disease, so funding them merely undermines our impulse to help the disadvantaged. By the end of the 1980s, Murray's characterization was being countered not only by critiques of his evidence, but by a round of books and reports *Presented as part of a Conference on Strengths and Potentials of Adolescence held by the Committee on Public Health of the New York Academy of Medicine March 8, 1991.
Vol. 67, No. 6, November-December 1991
584
M.A. BAILIN
presenting their evidence of the effectiveness of social programs, particularly those for children and adolescents in poverty. These new distillations began to offer us a glimpse of programs and principles that appear to succeed. The most famous of these volumes is Lisbeth Schorr's Within Our Reach.2 However, the new wave also includes Joy Dryfoos' Adolescents At-Risk,3 the American Psychological Association's 14 Ounces of Prevention,4 the National League of Cities' Children, Families, and Cities,5 publications of the Committee on Economic Development and the Children's Defense Fund, the W.T. Grant Foundation Commission's The Forgotten Half,6 and many, many others. All of them mine two decades of poverty fighting in an effort to discover what it is that does work for poor children, and they reach some common conclusions. First, they conclude that some programs do indeed seem to work. Within Our Reach describes 17 in some detail from the fields of family support, child welfare, and education. Included in the list are such nationally acclaimed efforts as Head Start and I Have a Dream, and less well-known ones such as Homebuilders in Tacoma, Washington. Adolescents At-Risk surveyed a much wider group of nearly 100 programs judged to be successful in the fields of delinquency, substance abuse, teen pregnancy, and educational remediation. Second, these researchers all tend to see common elements among effective programs. They are comprehensive, intensive, and flexible; they provide youth a great deal of individualized attention; they deal with the child in the context of family and community; they are characterized by strong management; they take a long and developmental view of providing help. The third common feature I see in these reports has more to do with tone than substance. They are optimistic. In her introduction to Within Our Reach Judith Viorst rejoices that we now have a blueprint to overcome what she calls a sense of "helplessness and hopelessness" about rescuing adolescents in poverty. In this spirit of optimism, Schorr contends that a "persuasive body of research and experience shows that the knowledge exists on which to build the interventions that will significantly improve outcomes among children and youth at-risk, including the children who are Truly Disadvantaged." Dryfoos agrees, declaring that "enough is known to greatly improve the potential life course for high-risk children in the United States." As president of an organization whose mandate is to locate and to test effective strategies for disadvantaged youth, I am naturally sympathetic to the considered opinion that concludes that such strategies are worthwhile, that Bull. N.Y. Acad. Med.
UWHAT WORKS WORKS
WHAT
585
they provide the lessons required for action, and that we are in a position to act decisively. However, I am concerned about the current tide of optimism that is attempting to replace the earlier bout of pessimism. While I have no qualms about the common-sense truth that some programs help high-risk adolescents and that they share some general features, I am as troubled by the assertion that we have our hands around some "cure" as I am by the earlier notion that social intervention is spreading the "illness." Unlike medicine, social policy research is at an incipient stage of development. We have little solid theory to rest upon, and our methods are primitive. We still have a hard time agreeing about the problem. One observer suggests, borrowing from medicine's history, that we have only recently started washing our instruments. Even the findings from our greatest success stories are much thinner than they might first appear. My reading of the research literature suggests that even for many of the best studies there is more statistical significance than actual significance or change in people's lives. Much of our confidence in Head Start rests on data from the Perry preschool experience involving just a handful of children; Job Corps' often-invoked success is based in large measure on crime that was not committed by young people while they were in the program; attrition in many of our most heralded efforts hovers near 50%; most successful programs touted in the reports I mentioned are small and idiosyncratic, and are yet to be seriously evaluated. To my mind, and this is supported by Public/Private Ventures' investigations, relatively few programs have been shown to work, even relatively few so-called exemplary programs work, and all too often those that do work at all produce only modest aggregate effects and uneven subgroup effects for the variety of participants who enroll. My concern about the state of our knowledge about what works is heightened because developing knowledge is only one fraction of the battle. The other fractions are even larger, pose greater challenges, and, until these are addressed squarely, bits of evidence about particular programs will neither inform the whole campaign nor provide the foundation on which we can move forward to assist large numbers of at-risk adolescents gain productive adulthood. THE LARGE CHALLENGES
In our work at Public/Private Ventures, three major challenges emerge from examination of "Why don't we know better about what works and what doesn't?" There is a clue in the two positions I referred to a moment ago: Vol. 67, No. 6, November-December 1991
586
586
BARN M.A. BAILIN
pessimists say that large public programs are not powerful enough to produce positive aggregate results and may even hurt people; optimists say that small, idiosyncratic programs do help some people; yet neither really offers convincing evidence to support their positions. I suggest that this is so because, in the public policy arena, we pursue program development and research without even a crude theory about what youth need to be successful; we test individual program components that address discrete fragments of a youth's problems rather than the whole youngster with all his needs; and we fail to establish convincing standards of success or how to transfer successful programs throughout the field. Therefore, I think we face three challenges. DEVELOP THEORETICAL FOUNDATIONS
First, we need to develop that crude theory of what it takes for a youth to have a reasonable chance of being successful -crude "because policy development, in the final analysis, is essentially a political process, and "reasonable chance" because even our best efforts are likely only to convert crises to problems, not to eliminate them entirely. My conviction about need for a theoretical foundation for our efforts emerges from Public/Private Ventures' experience and that of others working with youth. During the late 1970s and early 1980s, programs were developed and tested that addressed discrete pieces of a young person's experiencelack of work experience, lack of basic academic skill, lack of occupational skill, poor work attitudes, teen-age parenting, antisocial behavior-one at a time, and with little recognition of the variety of youth needs based on their age and stage of development. We should not have been surprised that we found few long-term effects. Gradually, we came to recognize the need for-and to require in our own work-a conceptual justification of what we meant by successful development based on theory or evidence for all the employment programs we support or evaluate. This may slow the pace of our work, to be sure; but quick fixes yield programs like the Family Support Act and the Job Training Partnership Act that have no theoretical or conceptual basis, even though we judge them as though they had. For us, a theory of what programs need to do to help at-risk youth become successful adults is beginning to emerge: They need to provide real opportunities -appropriate jobs and/or training experiences; they need to provide access to those opportunites by proffering everything from guided referrals to transportation to the suburbs; they need to Bull. N.Y. Acad. Med.
WHAT WORKS WORKS
WHAT
587
teach the personal and technical skills necessary for youth to take advantage of the available opportunities -and we now have a pretty good handle on what those skills are; and they need to assure participating youth the basic conditions of health, safety, adult caring and guidance so essential to develop those skills and to enable them to take advantage of opportunities. Indeed, there are exceptional youth who do not need all four components and exceptional programs that get results without providing them all. But these are typically boutique projects that have not demonstrated that they can be brought to scale, and good public policy must aim at addressing the needs of significant numbers of people. At first glance, these four components seem obvious enough and feasible to implement in programs. In fact, they are difficult to achieve in practice: social and economic trends push them beyond the reach of many urban at-risk youth. The number and quality of job opportunities are fewer; ways to access them are less available; required skill levels are rising; and the basic conditions of life- according to unanimous reports from the field -are debilitating and getting worse all the time. The point, in sum, is that a firm theoretical foundation would drive us to address all four aspects of youth's needs in an organized, continuous youthserving system. Today there is no system, and most programs focus only on one, or at most two, of the basic requirements. That is why their results are often so weak. Early interventions may provide basic skills, but usually do not deal with life conditions and seldom reduce the need for later assistance. Later interventions are therefore overmatched and asked to cover lost ground too late in the game. IMPROVE RESEARCH TARGETS AND PROCEDURES
Our second challenge is to develop a social research process that maximizes our chances of finding results -one that tests theories, not programs and one that tests them properly. As I have argued, testing programs that have just one or two components lead us nowhere. If youth need a range of services, testing programs that offer only a few and holding them to standards of long-term, life-changing success only dooms even the most promising to failure. Hence research must, from the outset, focus on program ideas with strong chances for success. In our testing procedures we are clearly still in adolescence ourselves. Funding often rides a policy wave or fad, so a research demonstration must start up quickly and get results fast. Medical research has long recognized the folly of such a procedure. There, knowledge development is pursued through Vol. 67, No. 6, November-December 1991
588
MA. M.A. BAILIN
repeated trials and modifications before formal testing begins, and very longterm follow-up is the rule. For social policy research, preliminaries and the long view are unheard-of luxuries. Little wonder we do not get strong results from our premature, short-term tests. And as for the government's role in developing and testing effective youth strategies, everything now militates against success. First, any demonstration responding to the four goals posed by my working theory-opportunities, access, skills, and life support-would require collaboration by at least several federal agencies, at a minimum, the Departments of Labor, Health and Human Services and Education. But, as we all know, funding streams do not flow together, mandates seldom converge, categories are either exclusionary and overlapping, and spoken languages are unique, to say the least. Second, a four-pronged approach requires, at the local delivery level, flexibility in responding to local conditions and resources and discretion in targeting and delivering services. But, as we know, that is not the norm. We now have traditionally trained professionals and conventional approaches that prize accountability for the expenditure of public funds. Third, demonstrating our theory would require delivery of intense, individualized services. But the reality is that public programs are required to serve the most people possible and to ensure equity, both in the face of falling dollar amounts. Finally, testing a conceptually sound system would involve both preventive and long-term services, while public pressure mounts for persuasive short-term payoffs, and funding of any service is unpredictable. TRANSFER RESEARCH-BASED KNOWLEDGE INTO THE FIELD The third challenge is to find ways to implement broadly programs that do demonstrate success. We simply do not have and desperately need public and bureaucratic support and structure that includes incentives and resources to encourage the transfer of effective programs and practices. Even when the demonstration/testing process does produce good results, our public infrastructure does not know what to do with them, beyond mailing out reports and holding occasional workshops. This produces little in the way of improved programming. It may produce better substantive policies since Congressional staffers do, more and more, read these reports. But in our experience, even that seldom improves programs and services for at-risk youth. There are simply no incentives or resources to ensure that effective programming is transferred and replicated. Our system puts too much focus on local invention and ownership rather than on effectiveness. And, of course, Bull. N.Y. Acad. Med.
WHAT WORKS WORKS
WHAT
589
there remains the issue of field capacity- staff terribly overburdened, insufficiently trained, and not very sophisticated in the replication and transferring of knowledge. The example of our Summit Training and Education Program (STEP) program is illustrative. It is a well-researched model with strong short-term impacts serving an underserved group of low-income young teen-agers with educational deficits that local service deliverers were required by federal legislation to address. It offers localities a tested, packaged, relatively cheap way to comply. But no mechanism in place enables localities to seize such opportunities. It took a specially planned effort by Public/Private Ventures, an intermediary, to market the model, to get the various actors together to provide local support, to train the staff and teachers, to provide the curricula, to show the localities how to run the program, monitor operations, and begin to construct a national network of STEP programs. Institutionalizing this replication function is clearly necessary. We may be seeing some movement in this direction in the fields of employment and training. Senator Simon introduced an amendment to the relevant federal law, the Job Training Partnership Act, last year that would have set aside a fund for replication. And the Departments of Labor and Health and Human Services have shown increasing concern about staff training in the use of model programs. But we still proceed as if good programs can always be a product of local good will and intention. Our experience shows that is not the case. Local people have neither the time nor the resources to develop new curricula, to articulate new approaches to learning, to rationalize and to document new ways to break down and to analyze jobs. Resources to do these things should lie at the national level, as should the resources to test them and to teach them. Our federal structure needs to see the proliferation of effective programs as one of its major responsibilities. Before moving on, I want to leaven the discouraging news just reviewed by noting that we see progress in meeting all three challenges. Funders, policymakers, and social observers increasingly recognize the necessity to construct and to test basic theories about what works, to support sensible interdepartmental demonstrations of approaches with solid conceptual underpinnings, and to develop a science of implementation that includes both structures and incentives. And though public programming and policy do not develop as rationally as we might wish, we see the legislative process paying increasing attention to evidence. Buttressing that trend is very important; it is our best hope. Without Vol. 67, No. 6, November-December 1991
590
590
BARN MA. BAELIN M.A.
evidence, and sometimes even with it, the same facts are open to different interpretations and conclusions by people governed by different political convictions, different public agendas. We need something more solid to go on. WHAT WORKS
Despite the deficiencies I have discussed, we have learned some things that can be useful now, both conceptually and practically. At the conceptual, or "broad lesson" level, what Public/Private Ventures is finding reflects consistently what Liz Schorr, Joy Dryfoos, and others have documented. One, we have found that successful interventions for at-risk youth address multiple needs simultaneously, not singly or sequentially. They are, in short, comprehensive and intensive. They overcome fragmentation through staff versatility and flexibility and by active collaboration across bureaucratic and professional boundaries. Two, we have found that successful programs are typically initiated and sustained by exceptional leaders-not because those interventions require exceptional people to implement them, but because our public funding and regulatory environment does so little to encourage, nurture, or sustain such programs. In fact, just the opposite. We need leaders to start and to maintain these programs in spite of the funding and regulatory environment. Three, we have found that these interventions focus as much on fostering a caring, supportive environment as on delivering professional services-not simply for warm-hearted reasons but because many at-risk youth are unreceptive to professional services until they have a sense they are cared for, and trust the adults who staff the invervention. Four, such interventions are usually inventive in the way they deliver services -they do not resemble the usual menu. Five, successful interventions constantly strive to connect youth to the larger society through understanding it, being involved with it. They do not take the safe route of building a program environment that works well so long as the real world does not intrude. They do not build a boundary around the program and, when finished with the youths, dump them into the real world. Let me briefly describe a few real programs we know well that exemplify these principles in practice for different stages of adolescence. For 10- to 14-year-olds there is I Have a Dream (IHAD) the scores of programs that echo Eugene Lang's model, which offers sixth graders a postgraduate tuition guarantee, sponsorship by a powerful financier, and, most critically, constant personal attention by a school-based project coordinator. Bull. N.Y. Acad. Med.
WHAT WORKS
WHAT WORKS
591 591
We spent time last year examining sixth- seventh- and eighth-grade "dreamers," as they are called, in three District of Columbia schools. Starting out skeptics, we finished the first year's look believing that IHAD shows real promise. Feeling special turned out to be an important motivator. The dreamers all spoke of basking in the attention they got from a successful person, as did their parents and their schools. The long-term, consistent, supportive counseling relationship of the project coordinator appears critical. This nonbureaucratic person responds flexibly and individually, with no restrictions on hours of service or what is and is not allowed. The job description says, literally, the project coordinator cares for these youngsters for six years. They need special tutoring; they get it. They need a ride to school; they get it. They need health services, glasses; they get them. IHAD's extracurricular activities involve the youngsters in the larger world and help them to feel a part of it. They go to plays, summer camps, big league ball games, a Japanese restaurant. Their horizons are expanded, they are enriched. After only one year of study, of course, we do not know if the District of Columbia's IHAD's early promise will hold up as youngsters scatter from their middle school to high schools all over the city and as overburdened coordinators possibly burn out. Nor do we know whether public institutions can enlarge or replicate the critical features and potential of this essentially private-sector effort. That is what we shall study and explore over the next years. The second program I shall discuss is for 14- to 15-year-olds. It's Public/Private Venture's STEP, which we designed, piloted, and tested in five cities with nearly 3,000 treatment youths. The model provides work experience and instruction in basic academic and life skills over two summers and continued contact with the program during the intervening school year. For the demonstration, we developed a new way to teach basic and life skills, both of which involve the youths in discussing real life -and introduced them to work at an earlier age than has been traditional. We are still looking at longer-term results, but short-term evidence is impressive: learning losses typically recorded over the summer by disadvantaged youths were eliminated for the treatment group, many of whom actually made gains over the summer in reading and mathematics; STEP youth increased their understanding of what they can do to handle their real life problems; and they came to appreciate and to like real work. But there it ends. Will regular schooling likely build on these gains? Probably not. And will public institutions widely utilize STEP? Not unless they continue to get assistance to do so-as Public/Private Ventures has Vol. 67, No. 6, November-December 1991
592
592
M.A. BAILIN M~~~~~~~~.A. BA..L...
provided to the nearly 100 communities that run STEP. The third program I shall describe is for 18- to 24-year-olds. It is the Center for Employment Training based in California. This is, I think, unique. It minimizes formal education, but integrates exactly what is needed into the practical skills training process. Each trainee selects the occupation he wants to be trained for, is assigned immediately a personal trainer who not only teaches occupational skills but also the academic skills necessary to perform that job -and helps the trainees get the support services they may need to stay in the program. The Center guarantees a job to every completer, a trick made possible by the organization's strong connections with employers, whose jobs they analyze down to every teachable component. The results? Right from the start the young people see real opportunities at the end of training. Early assessment and testing are dispensed with. Trainees get right down to work. They respond to the Center's innovative teaching, which works even with people who have little English. And they are bolstered by a caring environment in which staff members take responsibility to help youth address their life problems. What do these three programs, which deal with young people at different times of their lives, have in common? None started, or survive, dependent on a single and conventional public funding source. Each delivers services in an entirely different way. None insulate youths from the real world; each addressses the connections between the two and guides youths into developing their own. The two that deliver educational and/or technical skills needed substantial resources and time to hone their key techniques, and required more planning and resources to teach these techniques to others. In no instance has replication sprouted locally on its own. In sum, I think we are learning both broad principles and specific practices to help at-risk youth transition to adulthood. But I also think that the broad principles, once understood, are difficult to convert into effective practice at any reasonable scale. That requires an investment of resources to develop, test, refine, and modify-and then build out and transfer to other jurisdictions. Such investments are simply not possible within the current level or structure of public funding. Nor does it seem that we have the necessary political will to address the problems of at-risk youth seriously. THE ULTIMATE CHALLENGE: BUILDING CONCERN
Before concluding, I want to shift to another level of discourse and talk to you more as a worried human being and less as a student of public policy and manager of a research organization. Like most of you, I am worried about these youngsters and that society does not seem to share my concern. Bull. N.Y. Acad. Med.
WHAT WORKS
WHAT WORKS
593
For the first time in our history, children are the poorest age group in the population. And for those children growing up in poverty-in inner-city neighborhoods and housing projects that Nicholas Lemann recently condemned as being "among the worst places in the world to live" -life is full of stress, violence, and misery. Dangers are ever present, supports are few. Parents, themselves under great stress, can help to buffer their children against these circumstances, but it is often impossible for them to do so single-handedly. When they become incapacitated for one reason or another, their children are truly on their own. Missing from our inner-city neighborhoods-and as a matter of fact, as from most of our society-is the informal web of community, the loose collection of kith and kin who assumed responsibility in the past for helping to shape, nurture, develop, and discipline young people. Part of the erosion of this fabric of support and responsibility is attributable to the movement of black middle-class families out of the inner cities, but it is also part of a general weakening in our society of interest in childrenparticularly other people's children. Most parents barely have enough time to talk to their own children, much less their neighbors'. When it comes to children of the poor this indifference is damaging and widespread. Young people growing up poor in inner-city neighborhoods increasingly grow up in an unsupportive world -one in which they are cut off from much informal support in their immediate sphere, while simultaneously being abandoned at a national level by mainstream society. And that is really why people in our business go on these days collecting bits and pieces of research information, common-sense insights, moral intuition, compassion, whatever resources we can muster, all the while examining our progress and trying to redirect our efforts in ways I have sketched. In fact, there really is no viable alternative. Even lacking valid theories or a robust science of implementation, the needs of disadvantaged youth are so deep and pervasive that we must slog on. We do not have the luxury of waiting for a well-designed and fully detailed blueprint or for unassailable research evidence.
CONCLUSION I have called for better conceptual thinking, for better research, and for a science of implementation -but do I really believe that pursuing these goals will move us, as a nation, toward a caring society, one with a coherent family and children's policy, like those that many less rich nations now have? One that invests the resources necessary to ease the terrible problems of poor children and their families? Vol. 67, No. 6, November-December 1991
594
M.A. BAILIN
My own belief has been renewed recently by the rapid growth and popularity of mentoring programs and other initiatives that combine informal adult support, caring, and guidance with education and social services. But I fear that the public and policy interest in these initiatives is founded too much on the mistaken notion that they are cheap and easy. Public/Private Ventures' initial research on these approaches shows that they are, in fact, expensive and difficult to do well. Nevertheless, growing public interest in reconstituting a caring society an interest in part embodied in this wave of mentoring activity for at-risk youth -is encouraging. Its themes -responsibility, reciprocity, participation and interdependence -may be able to bring together diverse positions on the ideological spectrum, and may thereby contribute to generating the political will required to institute the reforms necessary if we are really to help adolescents who are at risk. Moving society to care about its young people may, in the end, be our biggest challenge of all and our greatest accomplishment. REFERENCES 1. Murray, C.A.: Losing Ground-American can Psychological Association, 1988. Social Policy, 1950-1980. New York, 5. National League of Cities; Children, Basic Books, 1984. Families, and Cities. Washington, D.C., 2. Schorr, L. and Schorr, D.: Within Our National League of Cities, 1987. Reach: Breaking the Cycle of Disadvan- 6. Youth and America's Future: The Willtage. New York, Doubleday, 1988. iam T. Grant Foundation Commission on 3. Dryfoos, J.: Adolescents At-Risk: PrevenWork, Family and Citizenship. In: The tion and Prevalence. New York, Oxford Forgotten Half: Pathways to Success For University Press, 1990. America's Youth and Young Families. 4. American Psychological Association; 14 Washington, D.C., William T. Grant Ounces of Prevention: A Casebook for Commission, 1988. Practitioners. Washington, D.C., Ameri-
Bull. N.Y. Acad. Med.
