Research resources.

Research Resources

CHAIRPERSON David Mechanic

The field of mental health services research (MHSR) focuses on the organization, financing, and delivery of mental health services (MHS) in a cost-effective way. like health services research but with a broader scope, MHSR is concerned with issues of financing, access, effectiveness, and how limited resources can be applied to achieve the best possible outcomes as measured by successful treatment, increased function, and improved quality of life. Mental health treatment and rehabilitation require not only effective psychiatric and medical care but also a wide range of social, educational, vocational, and housing services, which makes MHS coordination highly complex. Many of the services that severely mentally ill persons need are financed and administered by various agencies in the public and private sectors, by different levels of government, and by unrelated categorical programs. Understanding how to bring these services together for individual clients in a cost-effective way constitutes a major challenge to research and professional practice. Severe mental illness imposes enormous burdens on those afflicted, their families, and the community. Although the costs of mental illness for treatment and maintenance, in lost productivity, and in disability and suffering are so great, very little is invested in understanding how to reduce these costs and to use available mental health resources to achieve the best possible outcomes for individuals and communities. Individual professional practice may be guided by evidence from research, but organized systems of care, requiring billions of dollars each year, commonly proceed on the basis of trial and error and ideological positions with little of the feedback and systematic evaluation necessary for corrective action. Bureaucratic and professional prefer-

MEMBERS Joseph J. Bevllacqua Howard Goldman William Hargreaves James Howe Martha Knlsley Donald J. Scheri Qail Stuart Michael Bruce Unhjem PRINCIPAL NIMH STAFF LIAISON Thomas L. Lalley

669

ences reinforce patterns of care that are often insensitive to the heterogeneity of patient populations, the variability of course of illness, and the broad range of services that patients with mental disabilities require to achieve an adequate level of functioning and quality of life.

Needs for MHSR For many years it was assumed that the field of MHS delivery was too vague and unwieldy to yield systematic knowledge about cost-effective delivery of care. With Improved classification and measurement, methodological refinements, and increased availability of data on many aspects of care through a variety of Federal and State agencies, it is now easier to move the field forward in a manner useful to public policy and practice. A huge amount of information about promising forms of mental health practice, organization, and financing has been acquired that provides abundant opportunities to examine the effects of alternative programs and the most appropriate combinations of program components. Investment in MHSR has been paltry relative to the dimensions of the field and the research opportunities that exist. A small group of mental health administrators at the National Institute of Mental Health (NIMH) has nurtured a modest research program and constituency of investigators within a small services research budget that serves as a nucleus for future efforts. There is an urgent need to make a larger investment in this field, train more and better prepared investigators to bring the~necessary creative energy to help resolve urgent problems, and develop stronger interfaces between the research and practice communities on the one hand and systems of care and consumer groups

Downloaded from http://schizophreniabulletin.oxfordjournals.org/ by guest on August 24, 2015

VOL. 18, NO. 4,1992

670

The support of MHSR requires program stability as well as sizable new investments. Many of the complex services questions that plague delivery systems are generic issues that need sustained research to improve key areas of practice. As with results in other complex areas of biomedical research, important findings must be replicated under different conditions and with diverse populations to establish generalizability and applicability to various field settings. The field must establish long-term agendas consistent with the difficulty of the services issues and with the heterogeneity of conditions and patient characteristics. To stimulate superior research and attract outstanding investigators, funding agencies must demonstrate that they are serious about long-term support and that they will provide a stable research environment for talented investigators. Windows of opportunity made possible by new, short-term funding sources such as the block grant set-aside should be used to build long-term research agendas accompanied by a plan for long-term funding. The integrity of the research program, however, should not depend on set-asides, which are uncertain and viewed by some mental health organizations as competing with available funding for client services. The NIMH should develop an explicit plan to en-

sure long-term stability in its support of the MHSR agenda. In 1972 the Panel on Health Services Research and Development of the President's Science Advisory Committee urged a major expansion in health services research (Panel on Health Services Research and Development 1972). A study by the Institute of Medicine in 1979 recognized the pervasive involvement of many Federal agencies and made a number of recommendations on how Federal efforts could be strengthened (Institute of Medicine 1979). In subsequent years, health services research efforts expanded with particularly strong involvement of the National Center for Health Statistics (NCHS), National Center for Health Services Research (NCHSR), and, more recently, the Health Care Financing Administration (HCFA) and the Agency for Health Care Policy and Research (AHCPR). Effectiveness research now constitutes an important AHCPR initiative. Such studies as the Rand Health Insurance Experiment and the Medical Outcomes Study attest to the growing sophistication and usefulness of this research area. The quality of Federal cooperation affecting data acquisition and analysis and the launching of service demonstration and evaluation programs in mental health are major factors in the success of MHSR efforts. These efforts depend on data from a variety of research programs, including those of HCFA, the Centers for Disease Control (CDC), the Census Bureau, the Social Security Administration (SSA), the National Institute on Disability and Rehabilitation Research (NIDRR), and NCHS. NIMH should give special attention to the need for interagency coordination and cooperation at the Federal level to ensure efficient data acquisition and analysis on MHS issues. Also, multifaceted demonstra-

tions that involve innovations in housing, the administration of entitlements, and work rehabilitation programs require interfaces with such agencies as the Department of Housing and Urban Development, SSA, and the Department of Labor. It is a complex challenge to foster the necessary sensitivity to mental health needs among agencies whose major constituencies include only a minority of mentally ill persons. Stronger alliances among Federal agencies and with State and local agencies will allow more realistic and sophisticated initiatives and evaluations and will better serve the needs of mentally ill persons and their families. As service demonstrations and evaluation programs are launched, State and local agencies should be involved in all stages of the research process, from planning to dissemination. Initiatives may include such technical issues as ensuring that HCFA retains mental health elements in its public use data tapes or including mental health data items in important national or special surveys by NCHS and other Federal agencies. Conversely, initiatives may involve constructing experimental programs for evaluation in areas such as housing or vocational rehabilitation. Some of the needs are informational: many agencies dealing with mental health issues are unacquainted with mentally ill people and their special requirements. In some instances, essential data are not collected because of limited funding and competing priorities. Efforts must be made to develop more enduring interfaces. Major initiatives are essential to increase the range, sophistication, and capacity of MHSR efforts. There is a need to articulate the exciting research issues more dearly and to increase the rigor of the methodologies. A strong community of MHS researchers must


on the other. The present $45 million NIMH budget for MHSR and training and the improvement of State mental health statistical systems should increase to more than $200 million within the next 3 years. The Panel is fully aware of the limitations on resources and competition for them among many pressing needs. Yet it seems self-evident that serious responses to impending mental health challenges require that investments in services research and career training be increased severalfold.

SCHIZOPHRENIA BULLETIN

671

VOL. 18, NO. 4,1992

Services Research Process. Biotnedical and clinical research aim at understanding basic processes, with the goal of identifying and testing prevention and treatment strategies. Clinical trials seek to assess the efficacy and safety of new drugs, psychotherapies, and educational interventions. MHSR becomes relevant when interventions are applied in natural field settings. As a new drug or psychosocial treatment moves from the research site where it has been tested to other applications, a wide range of factors comes into play. The ways in which professionals and consumers perceive the innova-

tion, their willingness to adopt it, and its proper application all become problematic. Many social, organizational, and financial factors may intervene between an innovation, its acceptance, and its diffusion. Moreover, the innovation may not be applied according to protocol, making its effectiveness and safety less certain than appeared under more controlled conditions. Understanding the application and diffusion of technology and the factors that promote or impede successful implementation constitutes an important arena for MHSR efforts. Even when particular interventions prove cost-effective in MHS, their successful implementation depends on organizational and financial factors. A major shortcoming in MHS is the failure to develop adequate financial and organizational arrangements to support interventions with proven potential. Much of the disarray following deinstitutionalization derives from gaps and inconsistencies in financing, breakdowns in coordination among service elements, and the fragility of the organization of community care. Understanding how to develop the necessary financial and organizational structures and how they affect patients and families is crucial to the future cost-effectiveness of MHS. The investigation of these and many other issues outlined in the report of the research panels must be an enduring part of the NIMH research mission. Achieving this end will require sustained efforts to develop appropriate research environments and necessary cadres of researchers from a variety of disciplines. Targeting Research Priorities. A controversial issue in all science is the degree to which research should pursue investigator-initiated problems compared with research directed toward targeted objectives. The history

of science suggests that it is difficult to anticipate what knowledge ultimately will be relevant, and building on investigator-initiated research provides opportunities for the most talented researchers and most interesting and creative ideas. It might be argued that because health services research is specifically directed to current policy and practice, substantial targeting is justified. There is evidence even in health services research, however, that issues generated more by investigator interest than immediate policy have become important building blocks of future policy. Examples include early work on health maintenance organizations, hospital and physician reimbursement, and small area variations. MHSR should continue to maintain a balance between investigator-initiated projects and more targeted efforts such as specific requests for proposals or contracts. MHSR must give special attention to targeted service areas that are important but relatively understudied. Such targeted choices can be parsimonious and should be determined only after assessment of policy and practice needs. The reports of the Clinical Services Research (Attkisson et al. 1992, this issue) and Service Systems Research (Steinwachs et al. 1992, this issue) address these issues in detail. Whatever the targeted choices, it is essential to have mechanisms to ensure that the most creative and talented investigators have access to funding through investigator-initiated research proposals. There are important MHSR issues and areas of practice that require exploratory research. Often systems of care are too unstable to allow the type of research characteristic of the typical R01. The research in such areas must develop in exploratory and developmental ways that are adaptive to changing environments. Those in the


be developed and sustained to attack the most pressing services and policy research questions. MHSR "cultures" should be fostered in universities, in State departments of mental health, in major clinical settings, and in policy and health services research institutes. The field requires career structures to attract talented people at various stages of their careers, to develop their skills further, and to provide a stable financial environment for their research efforts. New mechanisms are necessary to encourage collaboration among researchers and MHS professionals and administrators. In short, a strong infrastructure must be developed to encourage and sustain the types of research essential to MHS policy and practice. This infrastructure requires leadership from NIMH but also from providers, State and local governments, and the academic and research communities. Services research must achieve the highest scientific standards, but this is unlikely unless supporting agencies demonstrate a strong and enduring research commitment. This requires support for methodological as well as substantive research. Health services research data must be able to withstand the toughest scrutiny.

672

NIMH should establish a program for developmental research grants in selected critical areas of MHSR that pose especially difficult and unusual challenges. These kinds of research often involve situations in which new problems are emerging faster than typical services research can reasonably track them. Examples include the impact of crack cocaine on emergency psychiatric services, changing financial incentives in psychiatric managed care, the effects of the acquired immunodeficiency syndrome (AIDS) epidemic on the functioning of the MHS system, and the impact of new nursing home regulations on alternatives for severely mentally ill persons. These are examples of issues of great importance to the mental health system that are receiving relatively little research attention. Researchers interested in these areas see little hope of funding through the conventional funding mechanisms because these areas are

too underdeveloped to result in a typical R01 application. Review committees should be asked to give special consideration to creative proposals of high program relevance in difficult and underdeveloped subfields. Research Environments. Health services research typically takes place in practice contexts with all of their uncertainties. Unlike the bench scientist or clinical researcher who can work in relatively controlled settings, the MHS researcher studies dynamic mental health delivery systems and the financial, organizational, political, and informational sectors in which they are embedded. MHSR depends on access to State and local MHS systems, including specific patient populations and treatment environments; data from Federal, State, and private sector information systems; and mental health providers. Such needs require understanding and cooperation between mental health administrators and the research community. Mental health care delivery systems operate in difficult environments and are buffeted by uncertain political and economic forces; thus, they sometimes are not as open to objective inquiry and evaluation as they might be. The administrator is naturally protective of his or her agency or program, whereas the researcher seeks to objectively ascertain the benefits and effectiveness of alternative approaches. In the long run, they share a common interest—the improvement of MHS— but in the short run their interests sometimes diverge. Administrators and researchers are also part of different "cultures" and respond to different expectations and reward systems. Administrators seek to enhance their programs, attract new funding, and improve their standing among competitive agencies. Researchers seek to publish research studies that meet peer

standards and attain recognition among other researchers. Mutual respect and trust must be developed in bringing these cultures together. Too often, researchers feel thwarted by administrators, and administrators feel that there is no quid pro quo for the time, inconvenience, and political risks involved in making their organizations available for research. Mechanisms to improve collaboration between service delivery systems and university researchers, such as the NIMH public-academic liaison (PAL) program, should be especially encouraged. Those engaged in such efforts must recognize the responsibilities of both parties but also be aware of the difficulties and the time required to develop the necessary trust. The PAL program is one important mechanism but tends to be directed to individual projects rather than a continuing and developing collaboration. NIMH should expand the PAL mechanism to include development grants for university-services system collaboration focused on the concept of developing structures for long-term research collaboration. PAL mechanisms should also be receptive to dual budgets, with funding allocated to both parties. In developing special requests for applications (RFAs) for MHSR, short-term windows of PAL collaborative opportunity should be used as a basis to develop long-term research agendas. The gold standard in health services research is increasingly the randomized controlled trial (RCT). Despite the large numbers of untreated and undertreated patients in public mental health systems that make RCTs feasible and ethical, relatively few are done. Clinicians and administrators often resist such trials even when feasible and ethical because they are not willing to substitute randomization for other criteria used to make refer-


field agree that such research is difficult to fund through the usual R01 and standing review committees. States should, of course, take some responsibility for early research efforts, because the MHSR problems commonly relate to local questions of importance to State services and policies. The Panel recognizes, however, the limited financial ability of many States to fund such efforts. Thus, it is recommended that exploratory work in MHSR be supported through the small-grant mechanism with clear communication to applicants and members of review committees that the process should be accessible and receptive to such proposals. Small grants should receive expedited review by the regular review committees, and early funding of the most meritorious proposals should follow. The responsiveness of the small-grant mechanism has eroded in recent years; it needs revitalization.


673

VOL 18, NO. 4,1992

Clinical MHSR Interface. Much of MHSR focuses on clinical practice in nonresearch settings. It asks questions about the ways patients are selected for various treatments, the effectiveness of complex interventions, and the impact of financial and organizational variables on the selection of treatment settings, choice of treatments, intensity of care, provider mix, and related issues. Although such research is different from clinical investigation, it requires considerable clinical sophistication. The health services researcher must understand diagnostic issues, processes of care, course of illness, iatrogenic effects, and many other details pertinent to mental illness. Moreover, clinical perspective and sensitivity are often important in conceptualizing and focusing key research questions. Thus, it is crucial to build strong interfaces between clinical practice and health services research in both research and training settings. Some clinicians from each of the key mental health specialties should receive MHSR training to ensure a proper balance in the research community among different perspectives. Experience shows that clinicians often recognize important issues that might be neglected by researchers with no clinical background. Discussions later in this report will address mechanisms to bring more psychiatrists, nurses,

and social workers into the MHSR field, which so far has attracted mostly psychologists, sociologists, and economists. MHSR—Health Policy Interface. MHS are substantially shaped by factors external to the mental health sector, including the structure of health insurance, health entitlement programs (particularly Medicare and Medicaid), and social welfare legislation as it affects such issues as disability, housing, and social services. Medical and psychiatric treatment of people with severe mental illness, prevention of secondary disabilities, and rehabilitation and habilitation depend on linkages among funding streams to ensure access to care and longitudinal responsibility for care. The development of effective mental health systems is complicated by the fact that they depend on a variety of categorical programs, which are administered by various agencies and levels of government. Many programs that affect mental health are developed for broader populations of clients, and these programs and their administrators are often insensitive to the special needs of severely mentally ill persons and the agencies that provide necessary services to them. Effective mental health policy requires understanding these entitlements and programs and the ways they interrelate at the point of service delivery, as well as at different governmental levels. Some of the programs of great importance to MHS, such as Medicaid and Medicare, have been topics of primary concern to health services researchers, but the mental health aspects are underdeveloped. Other important areas, such as income assistance, housing, and vocational rehabilitation, have been seriously neglected by MHSR. The ways in which such

programs come together at the service level and the opportunities and barriers they present for developing effective mental health delivery systems require investigation. The interface between mental health and other medical, social, and welfare programs involves levels of complexity that most MHS researchers are ill-equipped to investigate. Efforts are necessary to bring together mental health researchers and those who work in these areas from the general health service research field. More efforts must be made to acquaint new researchers with these important programs. Funding and technical support must be available to facilitate access to Medicare, Medicaid, and other large, important data bases and to create new data files on crucial issues affecting MHS. Efforts must be made to encourage NCHS to include more key mental health items on health care surveys (including the Health Interview Survey, the Hospital Discharge Survey, the National Nursing Home Survey, and the National Ambulatory Care Survey). Collaboration with HCFA's program is needed to help assess the effectiveness of care through its large billing files. Efforts must also be made to help mental health researchers access State information systems and to evaluate the reliability and validity of data items necessary for research and analytic work. It is recognized that increasing the involvement of NCHS in the collection of mental health statistics involves issues of financing that are difficult to resolve. NCHS should take such responsibility or, alternatively, the NIMH budget should be increased to cover the data collection costs. MHSR Centers. MHSR remains a very underdeveloped field, with only a small number of experienced researchers and few individuals who


rals to particular programs. In developing new collaborative relationships between researchers and delivery systems, there must be recognition of the importance of randomization and the responsibility for facilitating RCTs when they can be done. It may be advisable for NIMH to designate one or two development grants to help develop collaborative efforts for needed RCTs. More substantive aspects of RCTs are reviewed elsewhere in this report.

674

Recognizing the need to develop a stable, critical mass of researchers to address important MHSR questions that affect severely mentally ill persons, NIMH recently funded five research centers to study the organization and financing of care for severely mentally ill persons. One center received full funding for 5 years; four received partial funding. The MHSR centers perform a variety of essential functions. They bring together researchers from such distiplines as economics, sodology, operations research, and policy analysis to collaborate with dinidans and researchers from psychiatry, psychology, social work, and nursing on complex MHS issues that transcend any single disdpline or profession. In bringing such professionals together in research planning meetings and seminars, the centers allow for continuing interdisdplinary research training in a manner consistent with the complexity of mental health issues. This integration provides new opportunities for collaborative research; the centers can support pilot studies and preliminary data acquisition that help young

investigators to compete successfully for ROls. The centers also provide opportunities to create stable linkages among State departments of mental health, public mental hospitals, community mental health centers (CMHCs), and other important service entities. In addition to conducting research, the centers provide a forum for continued training and improved understanding among research workers in universities and the public mental health sector. The centers also provide rich interdisdplinary training opportunities, directed at the critical problems affecting severely mentally ill persons, for predoctoral and postdoctoral trainees, professional students in the key mental health disdplines, and for mental health managers and administrators in the local and State MHS systems. Interaction at the centers fosters interdisciplinary research, public mental health system-university cooperation, and increased mutual understanding. It also facilitates access to important research environments and collaborations. In addition to seeding new projects, the centers provide opportunities to develop pertinent data bases that can be shared by center participants and perhaps across centers as well. The development of a critical mass of expertise and the availability of funding provide opportunities to create new types of data files and to link data in ways difficult for individual investigators. Moreover, the coming together of MHS researchers provides the basis for developing technical support that transcends the work of any individual investigator. A major defidency commonly recognized in the MHSR field is the absence of mentors for researchers early in their careers and for dinidans who would like to devote a portion of their time to MHSR. The centers serve as

excellent contexts for mentorship and an effident means of organizing essential symposia, seminars, and interdisdplinary interaction for developing researchers in this new field. Four of the five currently funded centers have one or more NIMH training programs, and all five provide a context in which new investigators and mental health dinidans can develop skills and receive expert mentorship from experienced researchers in the substantive and methodological aspects of the field. The MHSR centers are critical at this point in the development of the field; the center program should be implemented aggressively and expanded. NIMH should provide stable and full funding for at least 10 such centers nationally. It is estimated that implementing this initiative in the next couple of years will require an increase in the center program budget from $5.5 million to $20 million. The program should be planned with a long-term developmental strategy and the promise of stable funding so that universities are motivated to allocate tenure-track positions to center activities. NIMH should consider ways of encouraging universities and States to invest in faculty positions and provide matching funds. Centers could successfully leverage their funds to obtain contributions from universities, State departments of mental health, private foundations, and other funding entities. Centers could serve as focuses for bringing together the expertise of their universities to address key MHS issues on a continuing basis. Exploratory and Pilot Research. As the competition for funds increases, study sections expect applicants to present research results supporting their applications. Individuals with preliminary ideas or interests in pursuing uncharted high-risk topics have


have a grasp of the complex systems that affect the delivery of MHS. Development of the field requires a critical mass of researchers ranging across health services disdplines who can examine central issues involving the broad range of relevant concerns and who can create research training environments for attracting and developing talented new investigators. There is also a need for research entities that can carry out research across service sectors, that can effectively bring together MHS researchers with clinicians and administrators, and that can develop continuing collaborative links with State mental health agencies, MHS programs, and consumer groups such as the National Alliance for the Mentally 111 (NAMI).


675

V0L18.N0.4,1992

Dissertation Research. An important means of expanding and improving MHSR is to fund talented doctoral students to carry out their dissertation research in this field. It is widely recognized that more value is achieved per dollar invested for dissertation research than for many other types of research. Dissertation research is usually supervised by a mentor expert in the area and by three or four other knowledgeable researchers. Although the quality of dissertation research has been limited by the meager funding available, the dissertation often establishes the area that the investigator will pursue after graduation. Other Public Health Service (PHS) agencies have recognized this oppor-

tunity, but dissertation research funding programs, such as the program at the former NCHSR, have become bureaucratized and less useful than they could be. A simple, adaptive mechanism is recommended to NIMH that will yield good value. Any funded MHSR grantee based in a major research university should be allowed to apply for a dissertation research supplement that the principal investigator (PI) can allocate to support dissertations in MHSR. Such supplements should involve a generic application and modest funding. The PI would be accountable for expenditure of the supplement funds but could administer these flexibly to encourage MHSR dissertation research and improve its quality. This mechanism could also be amplified to fund postdoctoral research and seed exploratory research of new investigators. Public Use Surveys and Other Data Bases. MHSR depends substantially on large information systems that provide an opportunity to track patients, procedures, and payments through the system of services and that supply a portrait of MHS organizations, providers, and related facilities. One such effort involves the inventories and surveys of the National Reporting System (Attkisson et al. 1992, this issue); other efforts include relevant surveys and information systems of government agencies such as NCHS, HCFA (Medicare and Medicaid data tapes), and administrative data systems such as those maintained by SSA. Still other important data may be derived from large-scale studies such as the Epidemiologic Catchment Area (ECA) program. Health services researchers do a great deal of important work using national and State data sources and depend substantially on well-documented data tapes provided by the

Federal Government or maintained by the Inter-University Consortium for Political and Social Research (ICPSR) at the University of Michigan. ICPSR, for example, maintains a national archive of computerized data on aging (NACDA) funded by the National Institute on Aging (NLA). This archive allows researchers to access easily a variety of data sets relevant to specialized interests in aging. MHS researchers need better access to important, well-documented mental health data sets. The inventories and patient surveys carried out by the National Reporting System should be made available to the MHSR community in public use data tapes in a timely and well-documented way. Moreover, major national studies (including the full data sets) such as those from the ECA program should be made available to the research community after an agreed-upon interval that allows the PI reasonable preferential access. Too many important mental health data sets are unavailable to researchers at the present time. NIMH should develop a strategy to disseminate public use of data tapes through ICPSR or other groups. A related problem confronts the investigator gathering data on MHS and clients in specific service systems whose study requires linkage of these data to data in related social service systems such as health insurance, income support entitlement, housing, and vocational rehabilitation. NIMH should take the lead in convening investigators with experience in crosssystem data linkage to plan the development of data system format, identifier, confidentiality, and other standards, and interagency agreements to facilitate such studies. NIMH should also engage in an active dialog with HCFA to ensure that important mental health data in its files are properly coded and accessible to the re-


little success through the regular research funding process, which increasingly favors detailed proposals following a line of established inquiry. This is a significant barrier for many MHS researchers who wish to address issues innovatively. Also, because funded programs of MHSR are relatively limited, new investigators lack a wide range of opportunities for piggybacking pilot data collection on an established project or program grant, as is often done in more traditional research fields. Furthermore, MHS researchers are commonly put at a disadvantage by study sections that do not understand the MHSR field and its needs. A variety of mechanisms should make seed money available for exploratory endeavors. One mechanism, already discussed, is the MHSR centers, which have some discretionary funding for young investigators and exploratory projects. A second possibility is the small-grant mechanism, previously discussed. A third mechanism is described below under 'Dissertation Research" but could be used more generally.


676

Responsibilities of Demonstration

Sites. As noted above, the research community has great difficulty gaining access to important demonstrations in the mental health field for evaluation purposes. Often demonstration site administrators have a stake in the idea being pursued but do not have the same commitment to evaluation. As suggested earlier, development of MHSR centers may help alleviate the problem, but the issue goes well beyond such initiatives. The Panel sees value in strengthening demonstration projects through rigorous evaluation, and it supports NIMH in its recent efforts to upgrade the design of these demonstrations. The Panel also supports NIMH in funding formative demonstration grants that can be further developed into grants using an evaluation protocol that meets rigorous research standards. In the long term, such a policy will yield more value for the money than the funding of less rigorous demonstrations, no matter how innovative. The proper function of demonstration monies is to focus on the issues of service development most likely to affect the quality of care, functioning, and welfare of persons with severe mental illnesses. Demonstrations and their evaluations should thus reach the highest attainable research standards. To evaluate how best to use demonstration dollars, NIMH should commission focused state-of-the-art papers to identify the most promising research innovations. This effort

should cast a wide net to ensure that the universe of promising innovations is well covered. Demonstrations and evaluations should then be carried out in a variety of States with carefully specified common approaches supplemented by additional interventions in particular instances. The demonstrations and evaluations should examine the effects of specified service innovations in various settings and with different client groups. Methodological Studies. The field of MHSR faces so many pressing and understudied issues that some people are impatient with investments that do not address immediate questions. The development of the field, however, must involve a long-range view with an enduring commitment to improve the quality of the science. Uncertainties of measurement and analysis limit the capacity of MHSR to answer many important questions rigorously. Much work is needed to improve the characterization of patients' illnesses and disabilities, and their changing courses. A common method for defining severely mentally disabled target populations and their various subgroups is also needed. Efforts have hardly begun to reliably describe, categorize, and measure the various service elements characteristic of MHS systems. Concepts and measurements of outcome are in an infant stage. Techniques to rate dangerousness, risk, competence, and consumer contributions to service delivery are underdeveloped. Data systems that can reliably track patients over time and through different services systems are greatly needed. Researchers currently can deal only crudely with the difficulties resulting from multiple sequential measurements, missing information, and varying reliability of records. Few studies examine health services implementation with suffi-

cient attention to medication dosage, compliance, and the intensity of related welfare and social services. Anyone with even the most cursory knowledge of MHS knows that such crucial terms as case management, capitation, comorbidity, dangerousness, and psychotherapy are used inconsistently, with little common meaning. Understanding how to characterize the strengths and liabilities of the client, the treatment provided, the organization of services, and financial incentives and disincentives requires a great deal of methodological work. Moreover, much more research emphasis must be given to mentally ill persons who are outside the formal system of care and to methods that allow longitudinal monitoring. Thus, it is recommended that NIMH issue special RFAs to improve measurement and analysis in the MHSR Held. To bridge the gap between the service policymakers and services researchers, NIMH should sponsor the development of applied methodologies in MHSR. State and local mental health planners and policymakers need tools for projecting needs for psychiatric beds, outpatient services, residential services, and other services and resources. They also need assistance in developing measures of program performance. Although NIMH has sponsored such activities in the past, a larger effort is needed to involve the research community in work that has a direct application to the needs of the service provider and planner. In that way, service providers and planners can be more involved in investigations and see practical results in this research area. Linkages. The successful treatment and rehabilitation of persons who have severe mental illnesses depend on linkages among a wide range of service sectors, including medical care.


search community and that research results relevant to MHS are included in policy considerations. NIMH should consider setting up a task force on the best applications of HCFA data elements, on the use of Medicaid tapes, and on ways of merging Medicaid and Medicare data files to assist MHSR.

VOL. 18, NO. 4,1992

Career Recruitment in MHSR. MHSR

is an underdeveloped field and has less prestige than work in health services research, health policy, and the social and behavioral sciences. If MHSR is to develop successfully, it must compete aggressively for talented young investigators. Although the field has begun to develop a critical mass in recent years with the assistance of NIMH, it faces difficult challenges and requires a vigorous recruitment strategy. In the past 10 years, faculty positions in the best research universities

have been limited, and many students in social and behavioral sciences, public health, and health services research have sought alternative careers. Postdoctoral traineeships have been important vehicles for bringing young investigators into MHSR and linking them to research opportunities. The lack of good faculty positions has made postdoctoral training attractive despite the uncompetitive stipend levels. In the coming years the labor market for faculty will change dramatically. Many of the professors who joined universities during the expansion of the 1960s will retire, creating a favorable job environment for new Ph.D.s. Postdoctoral training in MHSR, as in other areas, will appear relatively less attractive than faculty opportunities at strong research universities. In addition, in the past 10 years there has been a significant drop in graduate enrollments in some of the fields that provide MHSR talent; thus, difficulty in attracting outstanding young investigators to this new and difficult area of research can be anticipated. Because of the complexity of MHSR, few young investigators consider it as a career choice during doctoral training, although many have backgrounds in mental health issues in the social and behavioral fields. Specific training in MHSR has largely been at the postdoctoral or later career levels, which are the appropriate levels for most MHSR training. Before investigators engage in the interdisciplinary efforts required for sound MHSR, they must have a thorough grounding in the theory and methodology of a basic discipline. The fact that an investigator is a good economist, sociologist, statistician, or research clinician, as well as an interdisciplinary researcher, makes him or her

particularly valuable to the MHSR research team. Although the recommendations here focus primarily on postdoctoral training, it is clear that there is a virtue in initiating the training of some investigators at an earlier stage of their careers. A small number of individuals who identify MHSR as an interest should have a mechanism available to begin training before completion of their disciplinary preparation. In fact, it has been suggested that it is critical to begin the training of psychiatrists for careers in social research at the beginning of their residencies (Klennan et al. 1978). Furthermore, it is possible that more clinicians would be able to collaborate effectively as coinvestigators if their exposure and training began early in their internships, residencies, and other clinical placements. It is proposed that a program like the PHS Epidemiology Fellowship be developed for MHSR. In such a program, individuals who show great interest and promise would be given a competitive, multiyear (up to 4-year) stipend at an above-market monetary level to take with them to a training program. A trainee would bring his or her own resources to an established MHSR center to design a specialized educational plan. The training program would include research training, perhaps leading to an advanced degree, as well as clinical training. Program details would need to be developed. There is a serious danger in defining the MHSfieldtoo narrowly. Most of the outstanding researchers in the MHSRfieldreceivedstrong basic training in their disciplines and in mental health issues before becoming MHS researchers. Had the field been defined narrowly, many of these talented persons would not have selected MHSR as their primary career goal.


housing, social welfare assistance, and vocational rehabilitation. Successful programs depend as well on the involvement of private-sector organizations, advocacy groups, and a variety of nonprofit institutions, including foundations. Problems of linkage and coordination occur at the Federal level as well as in the research community. At the level of MHSR planning, NIMH should work toward greater collaboration with drug and alcohol agencies, as well as with other PHS entities. Perhaps most vital are the future relationships with the NCHS and HCFA, but there are important research opportunities with the SSA, CDC, AHCPR, N1A, and NIDRR in the Department of Education. NIMH should be a more aggressive advocate for mental health within the Federal Government and encourage the inclusion of mental health issues on the agendas of these agencies. More specifically, NIMH should encourage the development of a broader knowledge base on mental illness and should communicate research knowledge relevant to these agencies' programs as they affect the mentally ill. NIMH may be able to accomplish as much through efforts to develop mental health initiatives in these other agencies as through its own efforts.

677

678

Disincentives to MHSR careers are even greater among members of minority groups, who have few role models in this field. Moreover, because MHSR training occurs largely at the postdoctoral level, underrepresentation of minorities with Ph.D.s in the disciplines and with professional degrees further limits the recruitment of minorities. NIMH should give attention to career development among minorities and make special efforts to recruit more minority persons into the MHSR field. As noted in the other panel reports, research on ethnic differences and the ways in which mental health issues and services are perceived in different cultural groups is very much neglected. More researchers sensitive to these and other mental health issues as they affect minorities would help close this gap. Midcareer training opportunities, as well as facilitation of MHSR training for nurses and social workers (professions that include substantial numbers of minority group members), would help the recruitment process.

Mechanisms. The above recommendations can be facilitated by enhancing mechanisms currently in use at NIMH, by adapting mechanisms used elsewhere in the Alcohol, Drug Abuse, and Mental Health Administration, National Institutes of Health (ADAMHA/NIH), or by devising entirely new mechanisms. This section presents ADAMHA/ NIH grant mechanisms that are not currently used in NIMH that might facilitate MHSR. • F 33—National Research Service Awards for Senior Fellows provide opportunities for experienced scientists to make major changes in the direction of research careers, to broaden scientific backgrounds, to acquire new research capabilities, to enlarge command of an allied research field, or to take time from regular professional responsibilities for the purpose of increasing capabilities in health-related research. • G 07—Resources Improvement Grants are nonrenewable grants to establish a library or expand or improve libraries that have inadequate resources. • F 36—Visiting Scientists Fellowships strengthen research and teaching programs in the biomedical sciences for the benefit of students and faculty by drawing upon the special talents of scientists from other institutions. • K 07—Academic/Teacher Awards create and encourage a stimulating approach to disease curricula, attract high-quality students, foster academic career development of promising young teacher-investigators, develop and implement excellent multidisciplinary curricula through the exchange of ideas, and enable the grantee institution to strengthen its teaching program. • K 10—Special Scientific FYojects facilitate the use and increase the un-

derstanding of recorded information in fields related to health. The program enables qualified individuals to devote a period of full-time effort to the scholarly documentation, evaluation, and analysis of social, cultural, or scientific advances in various disciplines of the health sciences. • P 20—Exploratory Grants support planning for new programs, expansion or modification of existing resources, and feasibility studies to explore various approaches to the development of interdisciplinary programs that offer potential solutions to problems of special significance to the mission of NIH. These exploratory studies may lead to specialized or comprehensive centers. • R 21—Exploratory/Development Grants encourage the development of new research activities in categorical program areas. Support generally is restricted in level of funding and time. • S 15—Small Instrumentation Grants F'rograms are institutional awards to support the purchase of equipment costing within a specific range to be used by one or more recipients of research grants in these supported projects. The current MHSR peer review committee in NIMH developed out of the epidemiology review committee; perhaps a third committee is needed to review research and demonstrations. This committee would review all research and demonstration grant proposals, including exploratory or formative demonstrations (in which innovations are implemented for the first time), quasi-experimental demonstrations, and controlled services trials. This would allow NIMH and the field to depend on an established process for regular review of the entire range of research demonstrations. The committee should be composed of service providers and investigators


NIMH should support broad training relevant to mental health issues and not simply focus on the research imperatives of the moment. A strong health services researcher is likely to be a strong scientist in his or her discipline. Because MHSR is a field that depends on mature and experienced investigators, considerable attention should be given to advanced postdoctoral and midcareer training that brings in good researchers from the social and behavioral sciences and from the mental health professions. Stipends in the existing programs are too small to allow mature investigators with families to enter postdoctoral training programs. A midcareer special fellowship for new entrants into the field is needed.


VOL 18, NO. 4,1992

Human Subjects. MHSR often depends on direct assessment of clients and patients. Outcome studies are of increasing importance, as NIMH stimulates the field of clinical services research. In general, MHSR places human subjects at little or no risk but also provides them with little or no benefit. Observations in MHSR rarely consist of anything more invasive than an interview. Although some research subjects may find this upsetting, others find participation in survey interviews to be therapeutic. Safeguards can be developed to respond to problems that might be precipitated by research participation. Committees focused on human subjects appropri-

ately concern themselves with potential threats to informed consent due to mental impairments of the subjects in MHSR, but it is not appropriate to view them all as severely limited in their ability to give informed consent. Stigma about mental illness may make review committees too cautious, leading to the development of overly long and frightening informed-consent procedures. These procedures can have their most devastating effect on patients who are somewhat paranoid and who refuse to sign anything, even though they are willing to participate in an investigation. Failure to receive informed consent because of extreme protective measures is a major threat to obtaining unbiased samples of adequate size for important investigations. Investigators should be aware that, under some circumstances, there are approved precedents for obtaining unwritten verbal consent to participate in research. These should be considered before initiating a major study in which excessive changes in procedures for protecting human subjects could cause problems in sample acquisition and costly delays. The NIMH National Plan for Schizophrenia Research includes a lengthy discussion of these sensitive issues. The problems encountered in MHSR on severely mentally ill people are identical to those encountered by investigators working in the overlapping field of schizophrenia research. The Panel endorses the recommendations for the protection of human subjects contained in the National Plan for Schizophrenia Research. Personnel and Careers There is a critical need to recruit and support moreresearchersin the Held of MHSR. Barriers to recruiting young investigators include the stigma asso-

ciated with mental illness, the fact that MHS is a relatively new and underdeveloped research area, the difficulties associated with conducting research in the complex environments of mental health care delivery systems, and the fact that many talented young investigators have concentrated on biomedical and clinical research fields. Nonetheless, the value of research findings on the etiology, epidemiology, diagnosis, and treatment of severe mental illness depends on their dissemination and implementation with vulnerable populations through mental health care delivery systems. The work of many disciplines is directly relevant and necessary to MHSR, and considerable attention must be given to determining the best way to foster multidisciplinary research training and career development in the MHSR Held. Recruitment Incentives. The research environment in MHSR is complex and involves the interface of mental health, social, educational, vocational, economic, and political spheres of activity. Applying limited resources and innovations in an effective and cost-effident way requires the talents and contributions of a diverse group of professionals. MHSR needs to focus its recruitment efforts, therefore, on attracting a broad spectrum ofresearchersfrom such varied disciplines as psychiatry, nursing, psychology, social work, operations research, policy analysis, health care administration, economics, and sociology. This approach represents a departure from the traditional emphasis that NIMH has placed on funding predominantly biomedical researchers and requires innovative multidisciplinary recruitment strategies. Specifically, NIMH needs to acknowledge differences among the professional groups, such as physicians, nurses,


who would assess proposals for innovativeness and potential benefit for people with severe mental illness, as well as for scientific rigor. A committee of this kind could greatly assist NIMH in the exercise and development of its research demonstration authority. Some innovative services will be expensive to initiate and maintain throughout a period of research. Unless additional resources can be made available to the providers, these services will never be initiated. This is especially true of innovations that propose to develop and test a service that is not currently covered by some form of third-party coverage but could be if cost-effectiveness were established. It is recommended that, in addition to covering service costs in its demonstration grants, NIMH consider allowing marginal costs of new services as a research-related expense in sponsored research (e.g., R01 grants). These costs could be allowed if it were determined that an important innovation would not be available for study without authorization of additional resources.

679

680

Further, the stipend levels available for postdoctoral study are increasingly inadequate to attract outstanding people to the field. Researchers in MHSR are often older when they enter postdoctoral study and frequently have prior professional, clinical, or research experience. These are precisely the types of experienced persons the research field requires, but the stipends are often insufficient to allow these persons to support themselves and their families, particularly in high-cost areas. Attention must be given to ways of increasing or supplementing stipends to allow mature professionals to enter postdoctoral research programs. The benefits of actively recruiting from a variety of disciplines into the field of MHSR are numerous. The pool of potential researchers is large; there are increased opportunities to

recruit researchers from minority groups; and recruits with professionally diverse backgrounds can enrich research questions, methodology, and theoretical frameworks. These new recruits can encourage approaches that are broader in scope, more clinically relevant, and more sensitive to care-giving realities.

aspects of health care—attributes that are regarded as powerful and prestigious by society. In contrast, MHSR is carried out in natural field settings where there are complex individual, organizational, economic, and political factors that affect successful implementation. Frequently, these are viewed by novice investigators as barriers to research rather than as dimensions of the research problem. MHSR also involves activities concerned with caring for stigmatized individuals. Without the guidance and support of senior researchers, MHSR can appear to be a discouraging, overwhelming, and confusing field.

A number of factors discourage individuals from pursuing research careers, and these apply to the MHSR field as well. Personal disincentives include the high cost of professional education, which often leaves the new graduate with enormous debts; the lack of mentors to guide candidates into an MHSR career; the highly competitive nature of the Federal funding process; and the relatively low level of resources available for established MHS researchers. In addition, because MHSR is an applied field rather than a theoretical one, it lacks the status of other research areas; productivity in the area is often not rewarded with tenure-track positions in major universities. Thus, there are few incentives to attract individuals to MHSR and fewer training opportunities linked to career tracks that develop the skills of young investigators and provide them with a stable and supported environment to pursue their careers. Such support is necessary for them eventually to become mentors in their own right. In sum, MHSR has very limited educational and career infrastructures.

An additional problem is that many young investigators have little or no exposure to research in the MHSR field. Only 18 universities offer programs of study in MHSR. Most clinical training programs have minimal content related to working with severely mentally ill people, let alone organizational issues such as health care delivery systems. It is unrealistic to propose adding this content to existing clinical training programs and basic biomedical disciplinary training, because such programs already have a wealth of information to organize and integrate in a limited amount of time. It is suggested, therefore, that recruitment strategies for MHSR be focused on individuals who have completed doctoral study and are strongly grounded in the theory and application of their basic discipline.

The issue of the complexity of MHSR must also be addressed. Basic science and clinical research have many appeals to the novice researcher. There are more Federal funds and research mentors available. These areas also allow the young investigator to have considerable control over the variables involved in research. Finally, these fields are typically associated with the visible and curative

There are a number of benefits to this approach. MHSR requires a level of sophistication and integration of clinical and service issues that can best be accomplished by an established professional. Postdoctoral training would allow trainees to focus exclusively and in depth on MHSR with conceptual as well as applied perspectives. Finally, a postdoctoral emphasis would create an interdisciplinary pro-


and social workers, when determining stipends provided for postdoctoral study. The present postdoctoral stipends are based on years of postdoctoral experience. This credits new physicians for internship and residency experiences but penalizes groups such as nurses and social workers, who typically pursue a doctorate later in their careers. Unless they already have a Ph.D., nonphysicians are not credited for years of postgraduate clinical experience and are not appropriately compensated. This discourages social workers and nurses, who are frequently the primary service providers for severely mentally ill people and whose perspectives are greatly needed, from pursuing MHSR research training because they cannot afford to live on the present NIMH training stipends. These stipends also are not competitive with other offers that recognize the stage of career development and level of clinical expertise achieved by these professionals.


VOL. 18, NO. 4,1992

Career Training. An important aspect of effective recruitment is a stable and rewarding career structure for investigators entering the field. Researchers must be assured that MHSR offers challenging career opportunities with stable funding from Federal sources. Such support includes the expansion of research career development awards, long-term grant support, and commitment to research centers of excellence that are engaged in substantial training. The NIMH centers for the organization and financing of care for severely mentally ill people should be expanded to at least 10 centers nationally, and should be supported at levels that will allow them to undertake vigorous programs of multidisciplinary research, as well as serve sites for interdisciplinary MHSR training programs. As such, these centers could set up a national network for researchers in MHS, providing a research registry of senior investigators who might serve as enthusiastic mentors. Such centers might also facilitate the more active movement of researchers among various activities

and roles, including teaching, research, administration, and clinical practice. It would be desirable to allow individuals in MHS to move freely from one role area, such as administration or clinical practice, to another, such as research or teaching. It is dear that more flexibility and diversity need to be incorporated into traditional research careers if individuals are to sustain their enthusiasm and bring new perspectives to MHSR. MHSR activities should not be limited, however, to a small core of academic departments; rather, these primary sites should be the breeding ground for the eventual movement of researchers to rural and urban communities throughout the country. Issues of dissemination and implementation are critical to the field, and the MHSR effort must become involved with health departments and delivery systems across the Nation. This will require funding of researchers not located at major universities, as well as support for pilot and exploratory research in addition to traditional R01 research grants. Personnel Strategies. The implementation of these ideas requires recruitment and career incentives for MHSR that may depart from traditional norms. There are three basic needs: to increase the total amount of research monies available to MHSR, to create new grant mechanisms and expand existing mechanisms, and to articulate challenging questions in MHSR and find ways to communicate them to prospective interdisciplinary researchers. Some potential strategies include the following: • NIMH support of MHSR interest groups in a variety of professional associations (such as those in psychiatry, nursing, psychology, economics, public health, and health care admin-

istration) to expose potential researchers to the field and generate interest in it. • NIMH initiation of invitational conferences and open workshops for professionals from a variety of disciplines around topics of interest in MHSR, including organization and delivery issues, treatment, financing, needs of special populations, legal questions, research methodology, and evaluation. The goal of these conferences would be to highlight interesting questions in the MHSR field and to communicate them to prospective interdisciplinary researchers. • NIMH compilation of directories of individuals from various disciplines and from minority groups to promote their involvement in multidisciplinary studies in their areas of interest. Too frequently, contacts among researchers are insular and discipline-specific. • Increased NIMH funding of doctoral dissertations in the field of MHSR through the provision of individual awards and the granting of dissertation supplements to be allocated by selected funded MHSR grantees. • NIMH support of postdoctoral and midcareer training opportunities in MHSR through new fellowships. These would be designed to attract clinicians and researchers from key mental health specialties into MHSR in order to provide a proper balance and perspective. • NIMH enhancement of stipends that would make postdoctoral training in MHSR an attractive opportunity for potential researchers from various disciplines. This would include giving nonphysicians stipend credit for postgraduate work experiences that are highly relevant to understanding service issues and research needs in the field of MHS. • NIMH development of a mechanism to support State personnel in obtaining research training in univer-


gram focus that is essential to MHSR yet significantly lacking in existing clinical training programs. Earlier discussions in this report examined changes in the employment market for university scientists and growing indications of a future shortage in the social and behavioral science recruitment pool. Although new MHSR programs before dissertation research are not recommended, it is important that NIMH continue to support predoctoral training in the social and behavioral sciences relevant to mental health. Unless there is a pool of well-trained scientists to draw upon, MHSR training at the postdoctoral level will be deficient.

681


682

Research Strategies In MHSR there are several promising strategies that need further development, including formative services demonstrations, controlled services trials, and modeling studies of service systems. Development of these strategies requires that NIMH support more such studies, support methodological refinement, strengthen available technical resources, and strengthen institutional supports by facilitating interagency collaboration. Formative demonstrations, controlled trials, and modeling studies complement each other. In the typical formative services demonstration, a

new service model is developed and its potential is judged. Controlled trials provide clear tests of the cost-effectiveness of new service models relative to usual services. Modeling studies allow a range of service system effects to be examined for many more sites than is feasible in controlled services trials. It is recommended that formative demonstrations of new service models, controlled services trials of the most promising new models, and multisite modeling studies of the organization and finance of MHS be carried out within and across a variety of State mental health systems. Formative Services Demonstrations. Formative services demonstrations are designed to create new service models to respond to emergent problems, such as homelessness or alcohol and drug abuse among severely mentally ill persons. Formative demonstrations are often funded through legislation responding to public concern about an urgent problem. In formative demonstrations, specific service strategies may be tentatively formulated at the beginning and modified with experience. The purpose is to determine whether there is a service package that is feasible, acceptable to clients, and effective. Program implementation may be studied in detail. Sometimes effectiveness is formally assessed by comparing outcomes of clients in the demonstration program with outcomes of clients in usual care, but a fully developed controlled trial design is premature. Usually, only simple outcomes are measured. Research costs in formative demonstrations are modest. Formative demonstrations should move in the direction of greater rigor. The cumulative knowledge gained from a series of demonstrations will be greater if comparable populations

are identified, comparable measures are used, and similar questions are addressed. What kinds of clients accept services in the program? Who refuses the service or drops out relatively quickly? What is the percentage of clients still participating as a function of time since they entered the program (using life table analysis techniques)? What specific demonstration services were used by each client and at what cost7 What changes occurred over time (program entry, 6 months, 12 months) in client living arrangements, income sources, dependence on family support, use of restrictive or involuntary treatment, substance use, arrests and jail, symptom complaints, and general life satisfaction? What aspects of the program did clients endorse as most valuable, and what aspects did they criticize? What aspects of the program did staff rate as most valuable? How did staff describe the operating style of the program on a standard instrument designed for this purpose7 It is clear from these questions that standard methods for documenting the implementation and outcome of services for severely mentally disabled people need to be developed. These standard measures should be encouraged as a part of all formative evaluations. It is recommended that NIMH promulgate guidelines for evaluating formative services demonstrations and support development of improved evaluation methods. Guidelines should be widely distributed and discussed both with researchers and with State and local service providers to encourage strong services-research liaison. States frequently fund formative demonstrations, because the States administer the bulk of public MHS funds and are directly affected by improvements in service effectiveness or cost-effectiveness. Formative demon-


sity centers to stimulate the quality, quantity, and collaborative nature of MHSR. • NIMH priority for MHSR proposals that include participation from multiple disciplines and a combination of junior and senior researchers. • NIMH creation of special review panels for selected MHSR grants that allow for expedited reviews and the rapid allocation of funds. This type of funding process may be necessary to aid young investigators and research on "natural" experiments in MHS. • Initiation of special training programs to link public service providers and administrators with academic researchers and clinicians. These may be sponsored with State, university, or private monies and would have as their goal the increased identification, use, and dissemination of research findings on MHS. • Guest editorials and special columns in the journals of different disciplines to highlight MHSR questions and findings. This would provide an opportunity to disseminate MHS knowledge and research, as well as challenge and attract new investigators to the field.

VOL. 18, NO. 4,1992

Controlled Services Trials. When formative evaluations give favorable information about a particular service model, the model's cost-effectiveness compared with usual services should be examined by means of a controlled trial. Causal inferences sometimes can be drawn from quasi-experimental studies at single sites, where outcomes for the demonstration clients are compared with their own baselines or with nonequivalent client groups. More

often, a large number of competing explanations for the findings cannot be ruled out, and the evidence is not taken seriously. Modeling studies that use data from a large number of sites offer improved ways to draw causal inferences, but data sets from multiple sites rarely include an adequate range of either cost or outcome information. Remedying these problems in quasiexperimental prospective studies or multisite retrospective studies will often be as expensive and difficult as mounting a prospective controlled trial. Furthermore, evidence from controlled trials is compelling and thus weighs heavily in the analysis of policy alternatives. Despite its advantages, a controlled trial is often viewed with misgivings by service provider staff and management who have no experience with this research method. Techniques for dealing with the important ethical and logistical issues are unfamiliar to them, and the prospect of being accused of using their clients as guinea pigs is unsettling. Staffs know that researchers may not understand the complex constraints under which services are provided and that researchers may propose an unrealistic design or draw naive inferences from the results. These misgivings need to be addressed. University researchers, the staff of State mental health authorities, and service providers have different incentives that can undermine the basic trust required to negotiate an appropriate research design. If more controlled services trials are to be undertaken, potential participants must be allowed to gain a mutual understanding of the tasks involved. It is recommended that NIMH promulgate guidelines for planning and managing controlled services trials. Guidelines should be widely distributed and discussed both with researchers and with State and local service

providers to encourage strong services-research liaison. Under ideal circumstances, controlled trials follow a logical developmental sequence: Phase 1. A promising innovation is studied at a single site, with the service innovation compared with the usual service system. Phase 2. A very promising result from the initial trial leads to attempts to replicate it in other settings. Phase 3. An innovation with replicated evidence for cost-effectiveness is dismantled and enhanced, so that variations in the original innovation can be compared with usual services and with the original innovation to determine the most powerful components and to improve cost-effectiveness. Phase 4. One or more refined versions of the original innovation are taken to a multisite trial for consistent implementation and comparison with usual services. The purpose of the fourth phase is to confirm that costeffectiveness is maintained in different settings and with different population subgroups, that the innovation can be taught and monitored adequately to ensure consistent implementation under a full range of field conditions, and that unintended side effects do not seriously impair its overall impact. Findings from phase 4 will also indicate the range of settings in which the innovation is effective and serve as a basis for establishing quality assurance criteria. A phase 4 study can also serve as a major dissemination effort, because many service providers are being trained and monitored as they implement the innovation. A sequence of controlled trials comparable to the above is common in the development of discrete treatments such as a new drug. It has never been seen in full form in the development


strations are undertaken every time providers are funded to develop or implement a new model of service delivery to a problematic target group, although the quality of the ultimate documentation of the demonstration varies widely. It is recommended that NIMH explore incentives for State mental health authorities to use common measurement methods for formative demonstrations. NIMH technical assistance could enable States to provide consultation to their formative demonstration sites, and small service contracts given to evaluation staff in demonstrations could be incentives to assemble and report the evaluation information according to common standards. NIMH should gather and disseminate program descriptions from formative demonstrations funded by States. It is recommended that Federal funds be used to develop service demonstration capacity, with the longrange goal of supporting advanced demonstrations that meet rigorous research design standards. Such funding should also be used to support service costs in controlled trials of service models that have moved beyond the formative stage. Models that appear promising, and that have been sufficiently well-designed and described, should be studied in controlled trials in one or more sites.

683

684

An example of this developmental sequence is provided by studies of the integration of community treatments for severely mentally ill people. An assertive community team model has been considered a promising innovation since 1980, when Stein and Test (1980) reported the results of their second controlled trial carried out in Madison, Wisconsin. The cost-effectiveness measures employed in the second study were exemplary. In the assertive community team model, a majority of the contacts with consumers are in the community rather than in a clinic, and a multidisciplinary provider team with 24hour accessibility provides a full range of treatment and social support services. Controlled trials have now been reported on 11 implementations of assertive community team services, and results produced are generally promising. Interest in determining whether this model has even wider applicability continues to grow. Controlled trials involving dismantling and enhancement are being proposed and undertaken. For example, in one dismantling study funded by NIMH in San Jose, California, the full assertive community treatment model is being compared not only with usual services coordinated through officebased clinic facilities but also with an inexpensive program using paraprofessional companions. Although these individuals have no office base, they have access to professional backup staff and offer socialization, monitor-

ing, advocacy, and limited specific mental health treatment. In an enhancement study, an assertive treatment model combined with a capitated financing mechanism and a consumer empowerment emphasis is being compared with usual services. It may soon be time for a large multisite trial of the assertive community team model. Such a trial would be aimed at testing the limits of the model's applicability. A multisite trial would need to be carried out with rigorous monitoring and analysis of model implementation. Some important service innovations cannot be studied as within-site controlled trials. By their nature, these innovations are applied to all members of the target population in each community. Examples include any innovation that involves a large-scale reorganization in a service system. The Robert Wood Johnson Foundation/Department of Housing and Urban Development demonstration of the restructuring of mental health systems in nine U.S. cities is a case in point. It is difficult to determine the effects of whole-system innovations unless relevant data on several demonstration and control sites are available, which is not the case. A prospective controlled study of such innovations may be a better way to generate appropriate implementation, outcome, and cost data. The funding agency for such a demonstration would have to solicit applications from prospective site participants with the understanding that they would be organized into matched pairs by a set of criteria, with one site randomly selected for the innovation funding and the other as a control. Apparently no such trial has ever been attempted with MHS. Before it is attempted, more experience should be gained in measuring implementation, outcomes, and costs at single sites before and

after a system change. Such pre-post designs are feasible now, and their application to some contemporary innovations may yield valuable methodological developments. As with traditional controlled trials, whole-system trials ideally follow a logical sequence. A single-site, prepost study that shows an improvement in cost-effectiveness would ensure that methods were adequate to detect an attainable effect, at least under favorable circumstances. Such an outcome could be followed by replication pre-post studies at additional sites. With continued positive results, it might be useful to attempt the first paired-site randomized trial of a total service system reorganization. Specific research resources are needed for controlled services trials. These include an adequate supply of skilled investigators; accepted methods for measuring implementation, outcomes, and costs; and the commitment of service funding and program access. A supply of skilled investigators can be developed in part through experience. Single-site studies can sometimes be carried out by a beginning investigator with adequate consultation from experienced treatment trial investigators and health economists. MHSR centers potentially provide a setting for a junior investigator who wishes to undertake a controlled services trial. Multisite trials will require leadership from investigators experienced in controlled services trials, but participating sites can be led by less experienced investigators who can further develop their skills in the process. It is recommended that NIMH support the development of a larger cadre of experienced investigators not only through general training mechanisms, but also through special announcements for multisite trials and singlesite trials appropriate for beginning


of an MHS innovation. Implementing the full sequence will be a time-consuming process. Services research strategists must therefore be able to take a long-term perspective that identifies major promising innovations, understands their current state of development, estimates their ultimate potential value, and facilitates the next stage of research.


VOL.18, NO. 4,1992

Current measures of service implementation are especially weak. Ideally, measures of implementation would perform a dual function: they would document that the services being compared differ in the ways intended, and they would document how the services under study relate to the range of those usually provided in mental health systems. The latter function requires the collection of normative data on a large sample of service systems—development work that remains to be accomplished and will itself be a major research endeavor. Collaboration between service providers and researchers is crucial for controlled trials to be feasible. An investigator who is also the director of a service program is in a good position to integrate research and service needs successfully. Integration is also facilitated when a funding agency provides

demonstration funding linked directly to a strong research design. State mental health authorities are the largest MHS hinders; if the services trial effort is to expand to address the need for improved MHS, States must be major partners in this effort. The track record of successful collaboration between academic researchers and State mental health authorities on controlled trials has been very poor. Academic researchers tend to take long-term perspectives on the important research questions; shortterm results, on the other hand, are usually needed to inform the pressing policy questions to which State mental health leadership must attend. Academic researchers generally lack experience in service program leadership, especially at the level of the State mental health authority. They often have difficulty understanding the complexities of service implementation and the political concerns of public officials. Academic researchers and State leaders are aware that they are motivated by different aspirations and pressures. This makes it difficult to establish trust, which seldom develops without an extended period of collaboration and which usually requires more of a commitment than is legitimized by either the State government or the university context. The Panel recognized this as a major problem with regard to MHSR and a major impediment to improving the costeffectiveness of MHS. Simple answers do not readily present themselves. Current NIMH demonstration announcements provide some modest incentives for State-university collaborations. The NIMH PAL program has been especially useful. It may be valuable to consider issuing an announcement inviting applications for multisite, multi-State controlled services trials that explore new levels of

collaboration and can be models for the future. It seems wise to identify a research question seen as jointly relevant by mental health leaders in States that have expressed a consistent interest in research collaboration, and in which there are State research staff and academic investigators who are both willing and qualified to participate. Longer term consideration should be given to what will be needed for phase 4 controlled services trials. These will require extensive preparation, given the scale of funding required. If the innovation involves major fiscal and programmatic changes (e.g., capitation), it may require participating States to pass legislation authorizing and funding the study. NIMH should continue to provide occasions and incentives for research collaborations between State mental health authorities and university researchers. A multi-State, multisite controlled trial of an important service innovation can promote and deepen this collaboration. Modeling Studies of Service Systems. The variations in the organization and financing characteristics of MHS systems allow study of the relationship of these variations to system performance. Multiple regression, time series, stochastic processes, and other modeling techniques can be used to estimate the effects of service system variables, while controls are set to avoid potentially confounding factors. To conduct such studies, investigators must assemble data from complex information systems and public data bases. Linking across more than one data source is often necessary to examine the relevant research questions. Improved data resources must be developed to support such research. Among the needed actions are the following:


investigators with appropriate consultation. Clearer identification of accepted measurement methods for controlled services trials is needed. Several characteristics of severe mental disorders require special attention. These characteristics include their chronic, relapsing course; their disabling effects on working, maintaining social support systems, and acquiring housing; their effect of producing increased vulnerability to impairment from drug and alcohol abuse; and the wide range of public and private costs they engender in addition to the costs of treatment itself. These problems require developmental work on measures of program implementation, client outcomes, and client-specific costs. Investigators are currently developing new measures for implementing programs, adapting and simplifying measures of outcome that have been used in clinical treatment trials, and adapting cost assessment methods from general health services research.

685

686

Improving linkage among existing data sets. Data files from Federal agencies (e.g., SSA, Veterans Administration, HCFA, NCHS, Rehabilitation Services Administration, Department of Justice, ADAMHA) can be linked. So, too, can data from States, especially from State mental health authorities and State-Federal programs such as Medicaid. Creating new data sets. It is desirable to have comprehensive data on the organization and financing of public MHS systems throughout the United States so that researchers can examine the relationships of system characteristics to service patterns, costs, and outcomes. A description of the supply of MHS and providers (a picture of mental health markets) is a natural next step in the development of modeling studies. A uniform description of services, including innovations, needs to be developed. The current reorganization of the NIMH National Reporting System and its closer coordination with State mental health authority data systems are useful developments, as is the attempt to organize data reporting by geographic service system rather than by service program. To study the effects of service organization and finance on client out-

comes requires an even more ambitious development: a practical methodology that can be incorporated into routine information systems for measuring individual outcomes on an array of dimensions—for example, symptoms, social functioning, and quality of life. This will require major methodological work, including national field surveys to pilot and refine direct measures of outcome and to validate proposed indirect measures. Incentives should be created for the adoption of uniform data standards for client-level statistical data systems. The Mental Health Statistics Improvement Program (MHSIP) is a step in the right direction, but States must be given incentives to use the standard data definitions and elements proposed by MHSIP. A standardized approach for identifying severely mentally ill people is also needed. The Panel recommends that NIMH take the lead in coordinating an interagency process to facilitate the research linkage of all Federal and State data sets and information systems that bear on services and costs related to mental illness. It is also recommended that NIMH identify the new data sets needed to adequately portray the organization and financing of all public MHS in the United States. Finally, the Panel recommends that NIMH organize its National Reporting System by service system rather than by categories of programs. NIMH should develop a uniform taxonomy and dimensional description of service systems and service programs for mentally ill people that can be applied to existing systems and programs, as well as to innovations.

Dissemination There is a high level of public and professional interest in how MHS are organized and financed. A consider-

able body of research addressing these issues has developed over the last several decades. Unfortunately, there has not been a parallel development in mechanisms to disseminate these research findings. As the Nation's leading sponsor of MHSR, NIMH should assume a major role in dissemination. NIMH should establish a task force to develop a national MHS dissemination plan. The NIMH task force would be composed of leaders from the research and policy communities. Specialists from other fields with expertise in dissemination should be brought in for consultation. The task force would focus on developing cost-effective strategies for conveying promising research developments to mental health professionals, researchers, health policymakers, consumer advocacy groups, the general public, and other groups with a stake in the development of more efficient and effective mental health systems. To obtain valid information for dissemination, conferences of experts might be convened on selected topics. The Office of Medical Applications of Research (OMAR) at NIH has employed this strategy (Mullan and Jacoby 1985). OMAR has organized consensus development conferences to synthesize and report on medical topics to assist physicians with clinical decisionmaking. NIMH should consider using a similar strategy to develop an informed consensus on various key issues related to MHS. NIMH should look outside medicine in its search for dissemination strategies. The Departments of Agriculture and Justice have automated systems that disseminate information quickly to concerned parties. It would be useful to assess these systems for possible application to groups concerned with the mental health system.


Making existing data sets more useful. A uniform taxonomy of service, provider, and client characteristics must be established if existing data sets are to be rendered more useful. This includes reaching agreement on the dimensions that define a person with severe mental illness and on standards for the measurement of these dimensions. Data on client transfers and movements across service providers and service systems must be incorporated into existing information systems and data sets. Unique client or patient identifiers would enable these files to be linked more effectively.


687

VOL. 18, NO. 4,1992

Direct mailing is another strategy that has been used successfully to educate clinicians about important medical developments (Jacoby and Clark 1986). Special mailings on new findings in mental health care delivery could bring new information to clinicians in particular specialties. The feasibility of a newsletter for clinicians should be explored. Public education might best be achieved through the mass media. A recent community survey found that the public receives most of its health information through printed materials, television, and informal networks (Connell and Crawford 1988). Although the last medium is not amenable to direct intervention, the popular press and television are important and infrequently used resources for con-

veying information about MHS. Surveys of reporting on important medical topics (Winkler et al. 1986) and social science issues (Weiss 1985) have revealed that the popular press generally provides factual and balanced information. Mechanisms developed by NIMH to educate the public about the diagnosis and treatment of depression (Regier et al. 1988) might be useful in informing the public about MHS issues. NAMI and the National Mental Health Association should also be called on to assist in and provide opportunities for public education. NIMH should consider developing a mechanism to repackage significant MHSR findings in formats of interest to the media. A succinct summary of important developments in MHS might be generated and distributed to various media organizations. An easyto-read, brief fact sheet like the Age Page developed by NIA could report topical MHS issues in nontechnical language. Not only the media but voluntary mental health organizations, foundations, and advocacy groups would benefit from being kept up to date with such a fact sheet. Concerted efforts should be made to inform news organizations of important developments in MHSR. Certain journals with high visibility, such as the New England Journal of Medicine,

have dealt with the media in a marketing manner by coordinating information release so as to allow the press proper lead time to bring the story to a national audience. Similar approaches could be employed with important MHSR findings. Through increased newspaper, television, and radio coverage of MHS issues, the public will become more aware of the current problems with mental health delivery and the urgent need for exploration of strategies that promise improvement. MHS researchers are another important group to target for dissemina-

tion. Workshops and conferences should be organized on particularly promising topics. Highlights could then be published in appropriate journals or packaged in a format resembling the short reports published by the International Council of Scientific Unions Press (Ernester and Whelan 1987). A second strategy would be to develop a new MHSR journal devoted to republishing important research findings and review articles. A format such as the journal Behavioral and Brain Sciences might enrich scholarship. This journal features recent research and theory together with peer commentary. A clearinghouse modeled after the PHS's National Clearinghouse for Alcohol and Drug Information should be considered for the MHS area. The clearinghouse would collect, classify, store, and, most important, disseminate information on MHS. Information would be available in response to inquiries on specific topics. Such a resource would improve the transfer and development of knowledge. The timely transfer of MHS knowledge to policymakers is nearly as vital as its initial acquisition. Efforts should be made to disseminate MHS information at the Federal, State, and local levels. Elected officials with known interest in mental health issues should routinely be kept informed of promising developments in the study of mental health organization and finance. Efforts also should be made to increase the visibility of MHS reports in widely read policy journals such as Health Affairs. Dissemination to State mental health officials could be achieved through the National Association of State Mental Health Program Directors (NASMHPD), which has recently created a research institute that has given dissemination a high priority. NIMH should consider using this


Several dissemination strategies should be explored and developed. Because target audiences receive information through diverse sources, a multifaceted approach should be considered. For some clinicians, summaries of recent important articles and critical reviews are a preferred source of information (Williamson et al. 1989). National professional organizations such as the American Psychiatric Association, American Psychological Association, National Association of Social Workers, American Medical Association, and American Nurses' Association publish scholarly journals with large circulations. These journals are appropriate vehicles for publishing critical reviews of important topics in the MHS area. Journal space also should be sought for regular columns that address MHS issues. To reach clinicians who do not routinely read scholarly publications, NIMH should consider sponsoring regional or local conferences. These small conferences could provide presentations of relevant developments in the field and practical technical assistance.

688

Leadership for Mental Health Services Research: Rethinking Roles and Responsibilities MHS are delivered by multiple government and private entities, often

through conflicting and unrelated Federal, State, and private mechanisms. Complexity thus characterizes the topics and the settings of MHSR. This complexity requires a serious commitment by leadership in research and services arenas to resolve difficulties that limit the productivity of research efforts. For example, the Nation is in its third decade of efforts to create community-based care systems that decrease the need for and use of extended inpatient psychiatric care. Despite efforts to study these systems, questions remain about the effectiveness and feasibility of MHS for people with enduring and severe mental illnesses. Many questions about actual and potential client outcomes remain unanswered. In the organizational, political, and financial environments within which public service system leaders operate, the research questions are particularly compelling. Legislators want to know whether funds expended are achieving the desired effects in understandable and concrete terms. County and city officials face increasing pressures from service sectors such as education, transportation, and public safety, which compete with mental health expenditures. NIMH has formulated a national research plan for improving care for severely mentally ill people. As part of this process, the plan explored the approaches necessary to create and sustain a scientifically rigorous and pragmatically useful MHSR capacity. The concurrent needs of policymakers and dedsionmakers and the efforts of NIMH create an unusual opportunity to shape a more productive and mutually beneficial collaboration. These two groups (NIMH and the research community on the one hand, mental health administrators and the service community on the other) have complementary goals. Service system

leaders seek to develop care systems that function more effectively. MHS researchers seek to carry out studies on operating service systems that will advance the state of knowledge and inquiry not only in services but also in their research fields. Both groups have responsibility for contributing to the Nation's effort to provide comprehensive and effective services for people with severe mental illnesses. Effective leadership among researchers and service system executives is required to progress toward this goal. There has been too little collaboration and too much rhetoric. Many reasons have been put forth to explain why various groups cannot work together, but too few efforts have been made to demonstrate that they can. There has been far too little assessment of the impact of changes in service systems on client outcomes and too much political shaping of systems design. When data are lacking, opinion and ideology fill the gaps. The situation in the past decade might be characterized as an impasse between the apparent tyranny of pragmatism on the one hand and the seeming rigidity of research on the other. Three critical tasks are proposed for the development of improved collaborative leadership in MHSR: to identify and balance roles and responsibilities for leadership between public service systems and research communities; to create opportunities for research and administrative collaboration; and to develop more conceptual depth and rigor and more mutual understanding within both communities. While Federal policies have broad influence on the financing and organization of MHS, there are no Federal mental health policy leaders comparable to those at the State and local level. Therefore, primary emphasis should be given to the leadership provided by governors, mental health


newly formed organization to assist in or coordinate dissemination efforts. Another strategy for dissemination would be to require that State mental health authorities include dissemination issues as part of their State planning response to Public Law 99-660. Both NIMH and State authorities should invite consumers, families, and advocates to participate in formulating dissemination strategies and activities. NIMH should assist with the diffusion of technical knowledge required to transfer successful service innovations. The experience of Stein, Test, and colleagues at the Mendota Mental Health Institute in Madison, Wisconsin, provides an encouraging example of how assistance in training clinicians and administrators can facilitate the dissemination of a successful researchbased service system (Test 1992). The media obviously play a major role in the public's interest and awareness of MHS issues. Mental health issues often receive negative press or no attention at all. One of the barriers is the low level of knowledge and sophistication about these issues among journalists. NIMH should examine the value of funding a training program for outstanding journalists in a major university center deeply involved in mental health issues. Over a period of years such a program might produce a cadre of talented media writers and investigators with a commitment to responsible coverage of mental health issues. This approach has been used successfully in developing improved reporting in other fields of medicine and science.


689

V 0 L 1 8 . N 0 . 4,1992

commissioners, provider and advocacy associations, and others in the service system.

Secondary analysis of large data sets can be another significant source of service systems knowledge. Analysis of these data sets is helpful for understanding access, patterns of use, and overall implementation of a policy at the regional or State level. Secondary data analysis, however, usually does not provide the opportunity to investigate many of the subtleties of serving the various target constituencies of public mental health systems. State mental health commissioners and providers have sometimes felt that the research community is too focused on methodological issues and does not fully appreciate the realities they face. The research community, in contrast, has felt that service systems are tyrannized by pragmatism and lack the sophistication and organizational capacity necessary for a useful research agenda. The agenda must be rescued from this debate for MHSR to advance. In the past several years, State mental health commissioners have taken an increased interest in services research by developing a process to identify research priorities and by creating the NASMHPD research institute. The institute is designed to promote and conduct services research and evaluation studies that are not

feasible for individual States. The institute can also support multi-State analyses of service systems trends. The PAL initiative and the newly formed consortia of State and academic leaders can link the NASMHPD research institute to the academic community and inform this community of research priorities as viewed by State commissioners. State mental health commissioners and the governors they serve have an obligation to ensure that steps are taken to protect evaluation, training, and research activities beyond the tenure of these officials. Despite the different needs of the research and services communities, common themes and interests should continue to be identified, priorities established, and compromises worked out. The present level and amount of discussion among leaders of the two sectors are unprecedented. Sustained efforts, however, will be needed to move beyond discussion to implementation. It is recommended that NIMH and NASMHPD jointly convene a liaison committee to identify common interests, priorities, and means for sustaining a broader effort for service systems research. NIMH and NASMHPD should extend an invitation to foundations and professional organizations to participate in these deliberations. It is also recommended that regional and State MHSR consortia and public academic linkages be created and fostered. These groups should have as a major continuous agenda the strengthening of services and service system capacity, increased and improved research opportunities, and more effective applications of research findings. Creating Opportunities. Creating opportunities for MHS research must be an "inside" strategy. This means that research activities must become an integral part and a daily consider-


Leadership: Balanced Roles and Responsibilities. Leadership begins with the conceptualization and design of services and systems of care. Researchers need to be involved early in the process in cooperation with those responsible for actually providing the services (i.e., governors, mental health commissioners, and provider organizations). Incentives should exist for researchers and for the service system. In return for assisting providers with issues of conceptualization and design, researchers should expect to gain access to data, access to the populations being served, and the opportunity to identify important research questions and issues. The service system staff should expect that researchers will provide additional expertise in identifying important ideas and in addressing policy questions. Systems leaders need to realize that building in research and evaluative capacities may slow the implementation process somewhat, and that the design of a new policy or service may have to be altered to meet research needs. But if public systems are to make the most effective use of MHSR, researchers need to be involved in the policy process before all the difficult groundwork has been laid and crucial decisions have been made. As a tradeoff, policymakers can reasonably require that researchers recognize the need to move quickly to generate research findings, to forgo trying to be in control of all aspects of a service program they are studying, and to be willing to investigate research questions in which they are not particularly interested but which are of concern to the service system. Nearly every State mental health authority has some research capacity.

These resources, as well as outside investigators and consultants, have been grossly underused. Commissioners and others could make a significant contribution simply by recognizing and making greater use of the research capacities they already have. In so doing, they must be prepared for the eventuality that evaluative and outcome data may sometimes prove them wrong or suggest that innovations fall short of promises made. The leadership here derives from the courage to test assumptions and to alter original visions on the basis of empirical data.


690

mechanism at the State level, as required in Public Law 99-660, is a potentially excellent vehicle for integrating research findings in the development of State and local service plans. Personnel of successful model programs and demonstration projects should have opportunities to consult, help organize dissemination of results, and assist in broadened implementation. A good example of the value of this type of activity is the way the Program of Assertive Community Treatment (PACT) research findings and clinical expertise have been used to help guide and implement community MHS in Wisconsin. The research team, clinicians, academicians, and mental health authorities worked to integrate the findings into the design of services throughout the State. The State funded on-site consultation by PACT clinical personnel and contributed to the continuation of the original PACT program when outside research funding was unavailable. This mix of mutually supportive activities created a feeling of effective partnership. Another example of successful collaboration has been the use of ECA data for State planning purposes. In North Carolina, Duke University researchers made their data available to the State mental health division and assisted with estimating service needs and client diagnostic characteristics throughout the State. The result was that, for the first time, the State MHS plan was based on excellent research data. Too often, commissioners take little interest in proposals to outside agencies for support of MHS research projects and research centers. Their letters (and levels) of support are typically perfunctory and uninformed. The NIMH PAL initiative encourages a more active and positive posture.

State leaders are urged to help stimulate research and to take a lead role in the development of PAL proposals. More NIMH effort, however, is needed to support preliminary activities that can lead to a successful PAL proposal. NIMH should make funds available to facilitate linkages of research collection activities at the State level, with the goal of integrating research questions more effectively into the collection of management information data. NIMH should continue to foster and enhance MHSIP activities to improve State-level capacity to collect and use uniform data. NIMH and NASMHPD should provide technical information to States on how to use research resources and build research findings into their planning and operational processes. Better Answers From Better Questions. Leadership must go beyond setting the tone and creating opportunities. There is a need to support the conceptual thinking that creates better questions and sets a clearer course for research. The prevailing approach of both service systems and, unfortunately, much service systems research is developed largely from values, ideologies, and assumptions that remain obscure. The tyranny of pragmatism can have deadly effects on conceptual thinking for researchers as well as administrators. Service systems leaders seldom take the time to read and reflect on the larger enterprise in which they are engaged. Seldom do concepts, theories, questions, and controversies from the social sciences, clinical sciences, or other relevant disciplines penetrate the piles of paper and endless meetings. Yet research on diagnosis, labeling, stigma, various comorbidities, economic supply and demand dynamics, and the effectiveness of


ation within the service delivery system and that service researchers should actively participate in decisionmaking processes within these systems. For MHS to go forward, leaders must encourage evaluation and research as an integral part of the service system, especially at the State level, where systems are usually designed and funded and where data accumulate for a variety of purposes. The information systems now being updated and expanded by most State mental health authorities must attend to evaluation and research requirements as well as fiscal and resource allocation considerations. Financial requirements such as billing and reimbursement, as well as tracking of census and movement of patients, have largely driven the design of such systems in the past. Improved data on program content and change are needed. Commissioners and large urban and county providers can and should collaborate with researchers so that data needs of both communities can be accommodated. States need improved data to better understand the successes and limitations of program initiatives; providers need information to better manage their programs. Evaluation that goes beyond mere description of numbers of service units provided should be required in the local planning and funding process. Many plans and requests for funding contain data of questionable use that require considerable effort to collect. Technical assistance should be provided by NIMH and State authorities to improve the quality and usefulness of planning data. Commissioners must also make clear their serious commitment to getting good data, not just head counts. Researchers must be willing to offer assistance at this level in exchange for access to improved data. The newly developed planning

VOL 18, NO. 4,1992

The intuitive and imaginative thinking of an MHS systems administrator, linked with the perspective of a disciplined and creative scientist, can contribute to a level of conceptual thinking sorely needed in the field. All too often the questions focus on doing the job right rather than on first identifying what the right job may be. Leadership in this area needs to be embodied in partnerships between creative, risktaking, confident service system leaders on the one hand and skilled investigators and thoughtful scholars on the other. Representatives from both communities can profit from engaging in these ventures. Implications for NIMH.

This is an

especially opportune time to consider novel ways that leadership can be fostered in the conduct and application of MHS systems research. Several roles and responsibilities can be identified where leadership is most critically needed. Perhaps the most important is to transcend the turf battles between evaluation and research, between demonstrations and experiments, and between clinical and services outcome research. All these types of activities need to be encouraged. Evaluation is in essence a type of research and not by definition different in design, conduct, or merit from other MHSR. Every demonstration project provides opportunities for research in important service settings. Clinical interventions are typically inseparable from services. Bridging languages and research interests, with attention to strengthening the rigor of demonstration evaluations, must be a high priority. Random assignment to experimental and control conditions is one important methodology but it should not be the only standard by which MHSR is judged. Too often, it is assumed

that any research design short of this standard is unacceptable. On the other hand, the illusion that mere descriptive statistics constitute evaluation must be changed. Information relevant to and derived from service settings and clients is critical. Conceptual and methodological rigor is possible in these settings but takes time and resources to develop. NIMH must take on the role of assisting in the development of instrumentation, analytic techniques, and investigatory skill. At present, NIMH grant announcements often give insufficient lead time to develop collaborative agreements with university researchers and to make complicated site arrangements. NIMH should explore ways of increasing budgets for demonstration grants and extending the life of these grants beyond the present 3-year limit. Funding levels for demonstrations are too low, and funding cycles are too brief. It is recommended that NIMH consider incentive grants to States to develop public-academic linkages. Such a mechanism would lead to improved collaboration between the service and scientific sectors in the preparation of research proposals and the conduct of meaningful research. Conclusion. Neither the services research community nor the service systems community can excel in its tasks without the collaboration of the other. Each group needs to assume responsibility for improving the quality and productivity of mutual efforts. Effective guidance in this area begins with leadership, carefully conceived and focused on the very important task at hand. Executive Summary Organized systems of mental health care, costing many billions of dollars


new treatment technologies is relevant to decisions. Careful conceptualization should precede service systems change. Considered reflection on the underlying hypotheses and expectations related to major changes is often neglected. Purchase of services arrangements, for example, does not always increase competition among providers, lower the price of services, or spawn innovative programs. Recent research in Massachusetts calls all these presumptions into question. Seldom are these presumptions identified, much less tested, before the implementation of a major initiative. Mental health commissioners function in political and organizational climates in which there is little opportunity for them to question assumptions and explore hypotheses. Service systems researchers, on the other hand, can and should play a role in clarifying the conceptual underpinnings of systems change or maintenance. Hard questions about programs need to be brought to the attention of policymakers before decisions are made rather than left to evaluation studies conducted afterward. In Ohio, a long-term consultation with a researcher outside the organizational chain of command facilitated clarification of concepts before a fundamental reorganization took place in the State mental health system. Key elements were the early and continued involvement of the consultant over a span of years, the ability of the consultant to challenge in a collaborative way, the willingness of the leadership to be challenged, and the integration of research and conceptual perspectives into the decision process. As in the Wisconsin example previously cited, the researchers and the service systems administrators capitalized on their different perspectives and common concerns.

691

692

The members of the Research Resources Panel believe that the MHSR field is sufficiently advanced in concepts and methods to significantly enhance understanding of ways to deliver MHS more cost-effectively and appropriately. It is anticipated that the application of such knowledge will improve delivery of care, reduce disability and suffering, and contribute to better use of available resources through targeted financial and organizational strategies. There is an urgent need to upgrade the MHSR effort by making a larger investment in this field, training more and better prepared investigators to bring the necessary creative energy to urgent problems, and developing stronger interfaces between the research and practice communities on the one hand and systems of care and consumer groups on the other. The NIMH budget for MHSR and research training and for the improvement of State mental health statistical systems, now approximately $45 million, should be increased to more than $200 million within the next 3 years.

Recommendations Establishment of Stable Infrastructure for MHSR. • NIMH should develop an explicit plan to ensure long-term stability in its funding of the MHSR agenda. • NIMH should provide stable and full funding for at least 10 centers for the organization and financing of care for persons with severe mental illness. • Mechanisms should especially be encouraged to facilitate collaboration between service delivery systems and university researchers, such as the PAL program. • NIMH should expand the PAL mechanism to include development grants for university-service system collaboration focused on the concept of developing structures for long-term research cooperation. Such research mechanisms should encourage dual budgets, with funding allocated to both parties involved in the collaboration. Mechanisms for Facilitating the MHSR Agenda. • MHSR is a newly developing field that must function in a highly complex research environment. A variety of mechanisms should be available for providing seed money for exploratory and pilot research, including the research centers and the small-grant mechanism. • NIMH should establish a program for development grants in selected critical MHSR areas that pose especially difficult and unusual research challenges. Review committees should be asked to give special consideration to creative proposals of high program relevance in difficult and underdeveloped subfields. • In the development of special RFAs for MHSR, short-term windows of opportunity should be used as a

basis for developing long-term research agendas in critical areas of inquiry. • Exploratory work in MHSR should be supported through the small-grant mechanism, with dear communication to applicants and members of review committees that the process is accessible and receptive to such proposals. Small grants should be reviewed by the regular review committees, but both review and funding should be expedited. • A special mechanism is needed to facilitate research by new investigators. Any funded MHSR grantee (whether center, training grant recipient, or R01) based in a major research university should be allowed to apply for a dissertation supplement that the PI can allocate to improve dissertation research in MHSR. Supplements should involve a simple application and modest funding; such funding should also be used to support the exploratory research of postdoctoral and new investigators. • NIMH should support methodology development focused on services for the severely mentally disabled and severely emotionally disturbed target populations. This support can include investigator-initiated grants, a special announcement for methodological research, and an ad hoc consultant panel preparing commissioned methodological reviews. • NIMH should consider ways of encouraging universities and States to provide faculty positions, matching funds for centers, State office positions, and other new collaborative research arrangements funded by NIMH. • NIMH should consider establishing a new initial review group (IRG) to assess demonstration grant proposals, including exploratory, formative, quasi-experimental, and controlled services trials.


each year, are designed more by trial and error and ideologies than by systematic research and evaluation. MHSR, a relatively new field focused on the organization, financing, and delivery of MHS, offers an opportunity to understand how to apply limited resources to achieve the most cost-effective outcomes as measured by successful treatment, increased function, and improved quality of life. Despite the enormous national health problem posed by severe mental illness, little has been invested thus far in understanding how to better organize and finance care and how to dose gaps in service, reduce fragmentation, and alleviate the burdens of patients and their families.


VOL. 18, NO. 4,1992

Improvement of Data Sources and Analytic Capacity.

and data systems at the State level, with the goal of integrating research questions into the collection of management information data. • NIMH should continue to foster MHSIP activities to facilitate Statelevel capacity for collecting and using uniform data. • NIMH and NASMHPD should jointly provide technical information to States on how to use research resources and build research funding into their planning and operational processes. • NIMH and the States should create incentives for the adoption of uniform data standards for client-level statistical data systems. MHSIP is a step in the right direction, but States must be given incentives to use the standard data definitions and elements it proposes. • NIMH should coordinate an interagency process to facilitate the research linkage of Federal and State data sets and information systems that bear on services and costs related to mental illness. • NIMH should identify the new data sets needed to adequately portray the organization and financing of public MHS in the United States. • NIMH should reorient the NIMH National Reporting System so that it is organized by service system rather than by category of programs. NIMH should work with States to develop a uniform taxonomy and dimensional description of service systems and programs for people who are mentally ill that can be applied to existing systems and programs, as well as to innovations. Linkage Among Federal and State Agendes. • NIMH should give special attention to needs for interagency coordination and cooperation at the Federal

level to ensure efficient data acquisition and analysis on MHS issues. • A variety of Federal agencies have research and program responsibilities relevant to persons with mental illnesses. Such agencies include the NCHS, HCFA, SSA, CDC, NIA, AHCPR, and NIDRR. NIMH should play an aggressive advocacy role within the Federal Government to encourage development of a broader knowledge base on mental illness and on agency program responsibilities that relate to mentally ill people. • In the launching of service demonstration and evaluation programs, special attention should be given to coordination with involved State and local agencies at all stages of the research process, from planning to dissemination. • NIMH should work with States to develop its plan for a standardized approach to identifying people with severe mental illness. • NIMH should work with HCFA to ensure that important mental health data in HCFA files are properly coded and accessible to the research community and that research results relevant to regulation of MHS systems are included in policy considerations. • NIMH should take the lead in coordinating linkage of mental healthrelated data sets and information systems. • NIMH should consider constituting a task force on the best use of HCFA data elements, on the use of Medicaid tapes, and on ways of merging Medicaid and Medicare data files to best promote the MHSR field. • Efforts should be made to collaborate with NCHS in encouraging the inclusion of more key mental health items on the national health care surveys and with HCFA's program to assess the effectiveness of care through its large billing files.


• NIMH should sponsor the development of applied methodologies in MHSR. • NIMH should issue special RFAs to improve measurement and analysis in the MHSR field. • NIMH should explicitly encourage the use of ROls and small-grant applications for methodological and research feasibility studies. • Efforts should be made to help mental health researchers access State information systems and to evaluate the reliability and validity of data items necessary for research and analytic work. • Funding and technical support should be available to facilitate researcher access to Medicare, Medicaid, and other large, important data bases and to create new data files that address crucial issues affecting MHS. • NIMH should convene investigators with beginning experience in cross-system data linkage to plan the development of data system format, identifier, confidentiality, and other standards and interagency agreements to facilitate MHSR studies. • The inventories and patient surveys by the NIMH National Reporting System should be made available to the MHSR community in public use data tapes quickly and with good documentation. • Full data sets from major studies, such as the EGA program, should be made available to the research community after an agreed-upon interval, allowing the PI reasonable preferential access. • NIMH should develop a strategy for the dissemination of public use data tapes through the ICPSR and other groups. • NIMH should make funds available to facilitate linkage of research

693


694

Development of MHSR Personnel.

12. Guest editorials and special columns in the journals of various disciplines. 13. Conferences bringing together researchers from the general health services research field with those in MHSR.

• Recruitment strategies for MHSR should be focused on individuals who have completed doctoral study and who are strongly grounded in the theory and application of their basic discipline. • Special attention should be given to career development among minorities. • Oinicians in each of the key mental health specialties should have opportunities to receive MHSR training. • Differences in career paths among professional groups—especially clinicians, such as physicians, nurses, and social workers—should be considered in determining stipends for postdoctoral study. • Attention must be given to ways of increasing or supplementing stipends to allow mature professionals to enter postdoctoral research programs. • Attention should be given to advanced postdoctoral and midcareer training that brings skilled researchers from the social and behavioral sci- ' ences and from the mental health professions into MHSR. • NIMH should develop a program for MHSR comparable to PHS's Epidemiology Fellowship. • Mechanisms should be developed to allow individuals involved in MHS to freely leave one role area, such as administration or clinical practice, and spend time in another activity, such as research or teaching. • Career development efforts should include the expansion of research career development awards.

1. Support of MHSR interest groups in a variety of professional associations. 2. Invitational conferences and open workshops for professionals from a variety of disciplines around topics of interest in MHSR. 3. Compilation of directories of individuals from disciplines and minority groups relevant to MHSR. 4. Increased funding of doctoral dissertations in the field of MHSR. 5. Support of more postdoctoral and midcareer training opportunities in MHSR. 6. Enhancement of stipends that would make postdoctoral training in MHSR an attractive opportunity for potential researchers from a variety of disciplines. 7. Development of a special mechanism to help State personnel interested in research obtain research training in university centers. 8. Support for the development of a larger cadre of experienced investigators through the general training mechanisms and also through special announcements for multisite and single-site studies by beginning inves- • tigators who would be given appropriate consultation. 9. Special review panels for selected MHSR grants. 10. Foundation support for agendasetting conferences in MHSR. 11. Special training programs linking public service providers and administrators with academic researchers and clinicians.

• Formative demonstrations of new service models, controlled services trials of highly promising new models, and multisite modeling studies of the organization and financing of MHS should be carried out within and across a variety of State mental health systems. • NIMH should promulgate guidelines for evaluating formative services demonstrations and controlled services and should support development of improved methods for these demonstrations and controlled trials. Guidelines should be widely distributed and discussed, both with researchers and with State and local service providers, to encourage strong services-research liaison.

Research Demonstrations.

• Federal demonstration funds should be used to develop services demonstration capacities with the long-range goal of supporting demonstrations that meet rigorous research design standards. • NIMH should explore incentives for State mental health authorities to use common measurement methods for formative demonstrations. NIMH technical assistance could help States provide consultation to their formative demonstration sites. Small service contracts to evaluation staff in demonstrations could be used as incentives to assemble and report evaluation information according to common standards. NIMH should gather and disseminate descriptions of formative demonstrations funded by States.


• MHSR should focus its recruitment efforts on a broad spectrum of researchers from such varied disciplines as psychology, psychiatry, nursing, social work, operations research, policy analysis, health care administration, economics, sociology, and law.

long-term grant support, and commitment to research centers of excellence that are engaged in substantial training. • NIMH and other relevant agencies should support a wide range of mutually supportive strategies for developing personnel in MHSR. These efforts should include the following:

VOL. 18, NO. 4,1992

• NIMH should support methodology development focused on services to severely mentally ill persons. This support could include investigatorinitiated grants, a special announcement for methodological research, and an ad hoc consultant panel preparing commissioned methodological reviews. Human Subjects. • The Panel concurs with the discussion of human subject issues in the National Plan for Schizophrenia Research and endorses its recommendations on the protection of human subjects. Dissemination and Leadership. • NIMH should establish a special task force to develop a national mental health dissemination plan. The task force should be asked to devise strategies for disseminating research knowledge to interested constituencies, including the MHSR community, health policymakers, mental health professionals, consumers and consumer advocacy groups, and the general public. • The NIMH special task force should review the cost and effectiveness of various dissemination strategies, including the use of the mass media (editorials, news articles, television); conferences and workshops; video presentations; special mailings; and scientific, professional, and opinion journals, as well as collaboration with scientific, professional, and advocacy groups. The task force should

assess the dissemination systems in the Departments of Agriculture and Justice as possible models for new MHSR dissemination strategies. • The ADAMHA dissemination system should develop increased linkages within the Federal establishment and with State and local infrastructures. State mental health authorities, in their planning response to Public Law 99-660, should be required to address dissemination issues as part of the State planning process. • NIMH should consider more use of consensus conferences to review the state of expert opinion in key areas of MHS delivery. • NIMH should consider developing a new MHSR journal that republishes the most exciting and promising research findings with comments from members of the practice community. • NIMH should develop technical assistance teams experienced in specific health services innovations who meet with State and local officials and practitioners to assist in the diffusion of service innovations. • NIMH should develop brief, easy-to-understand announcements of service innovations for the general public and mass media. NIA's Age Page should be considered as a model. • NIMH should consider funding a training program for outstanding journalists in critical areas of MHS and policy. • NIMH and NASMHPD should jointly convene a liaison committee to identify common interests, priorities, and means for sustaining a broader effort for service systems research. NIMH and NASMHPD should extend an invitation to foundations and professional organizations (e.g., American Psychiatric Association) to participate in these deliberations. • Regional and State MHSR consortia and public-academic linkages should be created and fostered to

strengthen services and service system capacities and to increase and improve research opportunities and applications of findings.

References Attkisson, C ; Cook, ].; Karno, M.; Lehman, A.; McGlashan, T.H.; Meltzer, H.Y.; O'Connor, M.; Richardson, D. ; Rosenblatt, A.; Wells, K.; Williams, J.; and Hohmann, A.A. Clinical services research. Schizophrenia Bulletin, 18:561-626, 1992. Connell, CM., and Crawford, C O . How people obtain their health information: A survey in two Pennsylvania counties. Public Health Reports, 103:189-195,1988. Emester, L., and Whelan, W.J. Short reports: A new format for disseminating information from scientific meetings. FASEB Journal, 1:423-424, 1987. Institute of Medicine. Health Services Research. Washington, DC: National Academy of Sciences, 1979. Jacoby, I., and Clark, S.M. Direct mailing as a means of disseminating NIH consensus statements: A comparison with current techniques. Journal of the American Medical Association, 255:1328-1330, 1986. Klerman, L.V.; Morrissey, J.P.; and Goldman, H.H. Training psychiatrists in social research. Archives of General Psychiatry, 35:1469-1473, 1978. Mullan, F., and Jacoby, I. The town meeting for technology: The maturation of consensus conferences. Journal of the American Medical Association, 254:1068-1072, 1985. Panel on Health Services Research and Development of the President's Science Advisory Committee. Improving Health Care Through Research and Development. Washington, DC: Executive Office of the President, 1972.


• NIMH should continue to provide occasions and incentives for research collaboration between State mental health authorities and university researchers. One form this collaboration could take would be a multiState, multisite study of an important service innovation.

695

696


Friedman, M.B.; Herz, M.I.; Mulvey, E.P.; Snowden, L; Test, M.A.; Tremaine, L.S.; and Windle, C D . Service systems research. Schizophrenia Bulletin, 18:627-668, 1992. Test, M.A. The Training in Community Living model: Delivering treatment and rehabilitation services through a continuous treatment team. In: Liberman, R.P., ed. Handbook of Psychiatric Rehabilitation. New York: MacMillan Publishing Company, 1992. pp. 153-170. Weiss, C. Media report card for social science. Society, 22:39-47, 1985.

Williamson, J.W.; German, P.S.; Weiss, R.; Skinner, E.A.; and Bowes, F. III. Health science information management and continuing education of physicians: A survey of U.S. primary care practitioners and their opinion leaders. Annals of Internal Medicine, 110:151-160, 1989.

Available From NIMH

Free single copies of Special Report: Schizophrenia 1987 are available to requesters. The Special Report summarizes recent results of schizophrenia-related research. Topics covered include diagnosis, genetics, psychophysiology, biological studies, imaging, treatment, psychosocial issues, and theoretical issues. For the first time, the Special Report

will also contain nontechnical summaries to make recent research findings and issues more accessible to the general public. Readers who wish to receive a copy of the Special Report should write to Research Publications and Operations Center, NIMH, Rm. 10C16, 5600 Fishers Lane, Rockville, MD 20857.

Winkler, J.D.; Kanouse, D.E.; Brodsley, L.; and Brook, R.H. Popular press coverage of eight National Institutes of Health consensus development topics. Journal of the American Medical Association, 255:1323-1327, 1986.


Regier, D.A.; Hirschfeld, R.M.A.; Goodwin, F.; Burke, J.D., Jr.; Lazar, J.B.; and Judd, L.L. The NIMH Depression Awareness, Recognition, and Treatment Program: Structure, aims, and scientific basis. American Journal of Psychiatry, 145:1351-1357, 1988. Stein, L.I., and Test, M.A. Alternative to mental hospital treatment. I: Conceptual model, treatment program, and clinical evaluation. Archives of General Psychiatry, 37:392397, 1980. Steinwachs, D.M.; Cullum, H.M.; Dorwart, R.A.; Flynn, L.; Frank, R.;

Research resources in academic radiology.

New technology and resources for cryptococcal research.

Web resources for stem cell research.

Research Resources for Nuclear Receptor Signaling Pathways.

Resources, challenges and way forward in rare mitochondrial diseases research.

Allocation of Resources to Communication of Research Result Summaries.

Resources, challenges and way forward in rare mitochondrial diseases research.

Incorporating computational resources in a cancer research program.

Human resources for research: building bridges through the Diaspora.

Research and application of knowledge resources network for product innovation.

The louisiana clinical data research network: leveraging regional and national resources to improve clinical research efficiency.

Leveraging Online Learning Resources to Teach Core Research Skills to Undergraduates at a Diverse Research University.

Reply to Letter: "Inconsistent Data Resources Weaken the Quality of Research Results".

Ethics in psychiatric research : resources for faculty development and resident education.

A brief qualitative survey on the utilization of Yoga research resources by Yoga teachers.

Bio-objectifying European bodies: standardisation of biobanks in the Biobanking and Biomolecular Resources Research Infrastructure.

American Recovery and Reinvestment Act-comparative effectiveness research infrastructure investments: emerging data resources, tools and publications.

The ABC of E-learning: Utilizing E-learning resources for training and research in gastroenterology.

Computational approaches and resources in single amino acid substitutions analysis toward clinical research.

Missing variables: how exclusion of human resources policy information confounds research connecting health and business outcomes.

Outcomes research resources in India: current status, need and way forward.

Dissemination of health-related research among scientists in three countries: access to resources and current practices.

Cross border semantic interoperability for clinical research: the EHR4CR semantic resources and services.

Developing Educational Resources to Advance Umbilical Cord Blood Banking and Research: A Canadian Perspective.