J Chron Dis 1977. Vol. 30. pp. 257-259. Pergamon Press. Printed in Great Britain
THE CASE FOR MANAGEMENT RESEARCH FOR LARGE FIELD TRIALS JAMES E. GRIZZLE School of Public Health, The University of North Carolina. Chapel Hill, NC 27514 (Received
in rerised
form
SEVERAL very expensive
10 Octohn 1976)
evaluations of interventions in chronic diseases are now underway in the United States. More than one author has expressed the opinion that these studies are too expensive and that they sap the resources available for more fundamental research [I]. What is fundamental, like beauty, may rest in the eye of the beholder. However, even after discounting for biases the authors may hold, the issues they raise should not be ignored. Well controlled intervention studies in chronic disease are very expensive--so expensive, in fact, that it may be questioned whether every promising intervention can be tested by methods now generally accepted as being required for its rigorous evaluation. Rather than questioning whether these studies can be afforded, a more appropriate question is how can their cost be kept within reasonable bounds without sacrificing quality‘! At present, very little objective evidence exists on how to deploy resources to achieve the design goals of a study. If study designs are to be modified in ways that will reduce costs without destroying scientific validity, statisticians, epidemiologists, and computer scientists who are interested in chronic disease research, must commence systematic study of the whole process of organizing and maintaining exemplary studies of chronic disease. Statisticians have customarily approached the problem of increasing the efficiency of studies by seeking inprovements in methods of data analysis and in the mathematical aspects of study design. A good example of this type of investigation is given in [2] which examines the relationship between the level of risk of a subject and the cost of large scale field trials. Another example of a problem that is currently receiving considerable attention is that of interim data analysis. Significant problems remain in these areas. These and similar problems undoubtedly will continue to receive their fair share of attention. The equally important area of maintaining the quality and scientific integrity of a study has received very little systematic investigation. For convenience we shall consider both of these areas jointly under the heading of study management. It includes protocol development, forms design, data management, quality control of laboratories and of field work in general, training field workers, monitoring the quality of field work, and relationships with study committees and study 257
258
JAMES E. GRIZZLE
centers. In general, it encompasses all of those activities thought to be necessary for the collection of data that are of adequate quality for the purposes of a study. Obviously, statisticians will not bear the full brunt of all these obligations, but they should be among those most concerned about taking the appropriate steps to meet a study’s goals for data quality. Statisticians who are specialists in sampling and their collaborators have designed special studies in several areas to evaluate the contributions to the mean square error made by interviewer bias, respondent bias, sampling error and various aspects of field work. These studies have provided sound criteria which can be used to allocate the resources available in the most efficient way for some types of surveys. In contrast, one seldom sees detailed exposition, supported by objective evidence, on the most efficient way to organize and manage clinical trials. Greenberg[3] has given an outline of some of the ingredients necessary for a successful study but he does not go into the cost aspects of study management. At the present time, methods are adopted or discarded on the basis of intuition. This author has never seen an objective evaluation of the extent to which any of the study components thought to be necessary actually contributes to the mean square error or to any other measure of the statistical precision or validity of a study. A number of plausible explanations exist for the lack of systematic assessment of the components that are necessary for a successful clinical intervention study in chronic disease. These studies are a relatively recent development. Not all centers of education in the relevant disciplines have faculties with first hand experience in the design and conduct of such studies. Most of the knowledge about design and maintenance of long term studies is transmitted by on-the-job training and other informal methods rather than by the usual academic routes. Until recent moves by coordinating centers for several NHLBI financed studies to hold annual meetings, there was very little exchange of technology among different coordinating units. One has the impression that the wheel is continually being reinvented but never evaluated. Probably the greatest obstacle is that the methods of organizing and maintaining an intervention study of high quality have not been recognized as a respectable area of inquiry. This in turn results in a lack of financial support for investigation into how the methods now in use can be improved and resource allocation made more rational. Without systematic investigations there are no journal articles or books, which are necessary to stimulate further research and facilitate a more orderly transfer of knowledge. Consequently, the academic community has not granted this area a high level of respectability. Apparently it falls in the interface of management, epidemiology, and statistics. Cochran [4] mentions that persons who are successful in key directive roles in long term studies are different from the ‘ivory tower’ concept of scientist.
“He must be able to assemble a team of workers and to maintain good relations among them: he must obtain the cooperation of various administrators and their agencies and must handle a considerable amount of paper work. Scientific competence alone does not guarantee success in this type of research: some scientists are too shy. and others too quarrelsome, to meet the requirements.”
The Case for Management
Research
for Large
Field Trials
259
Obviously all the desired personality attributes cannot be taught, but this should not prohibit the systematic development of a useful body of knowledge. Most, if not all, of the ingredients necessary for the development of design and management of studies of chronic disease as an area of scientific study and academic training are identifiable. The first requirement is the recognition that there is a body of knowledge to be acquired and that it is capable of organization and further development. Persons who are successful in this area should be recognized as productive scientists and treated as such in the scientific community. This should include publication, in scientific journals, of their research into the methodology of management of clinical trials in chronic disease, and recognition of the value of this work in the academic community when appointments and promotions are made. Systematic development of the field cannot take place without some financial support. Centers which have contracts or grants to coordinate studies of chronic disease should be provided with some funds for investigation into the methodologies they use. Experimental comparison of different methods now in use to accomplish the same ends would be a good starting place. A budget of even l-2”,, of the total money being spent in intervention studies in chronic diseases should lead to the development of useful knowledge. Without this knowledge scientists in this field cannot objectively allocate their resources in optimum ways to control cost. The standards for study design and administration appear to be increasing as a consequence of growing sophistication among clinical practitioners, epidemiologists, and statisticians and of concern for patient’s welfare. It is beginning to be recognized that costs increase with increasing rigor of study design and the ensuing problem of maintaining the design quality over an extended period of time. These costs may grow to the extent that the desired levels of rigor cannot be achieved for many important studies unless data are accumulated to evaluate the cost benefits of various measures undertaken to maintain the study. It is time to recognize this problem and to develop some reliable recommendations for use in the field.
REFERENCES Marmot M. Winklestein W, Jr: Epidemiologic observations on intervention trials for prevention of coronary heart disease. Am J Epid 101 : 177-181, 1975 Sondlk E, Brown B. Jr. Sdvers A: High Irisk subjects and the cost of large field trials. J C’hron Dis 27: 177- 18X. 1974 Greenberg B: Conduct of cooperative field and clinical trials. Am Stat 13: 13- 17, 1959 Cochran W: Research techniques in the study of human beings. Milbank Mem Fund Quart 33: 131-136. 1955
THE CASE FOR MANAGEMENT RESEARCH FOR LARGE FIELD TRIALS JAMES E. GRIZZLE School of Public Health, The University of North Carolina. Chapel Hill, NC 27514 (Received
in rerised
form
SEVERAL very expensive
10 Octohn 1976)
evaluations of interventions in chronic diseases are now underway in the United States. More than one author has expressed the opinion that these studies are too expensive and that they sap the resources available for more fundamental research [I]. What is fundamental, like beauty, may rest in the eye of the beholder. However, even after discounting for biases the authors may hold, the issues they raise should not be ignored. Well controlled intervention studies in chronic disease are very expensive--so expensive, in fact, that it may be questioned whether every promising intervention can be tested by methods now generally accepted as being required for its rigorous evaluation. Rather than questioning whether these studies can be afforded, a more appropriate question is how can their cost be kept within reasonable bounds without sacrificing quality‘! At present, very little objective evidence exists on how to deploy resources to achieve the design goals of a study. If study designs are to be modified in ways that will reduce costs without destroying scientific validity, statisticians, epidemiologists, and computer scientists who are interested in chronic disease research, must commence systematic study of the whole process of organizing and maintaining exemplary studies of chronic disease. Statisticians have customarily approached the problem of increasing the efficiency of studies by seeking inprovements in methods of data analysis and in the mathematical aspects of study design. A good example of this type of investigation is given in [2] which examines the relationship between the level of risk of a subject and the cost of large scale field trials. Another example of a problem that is currently receiving considerable attention is that of interim data analysis. Significant problems remain in these areas. These and similar problems undoubtedly will continue to receive their fair share of attention. The equally important area of maintaining the quality and scientific integrity of a study has received very little systematic investigation. For convenience we shall consider both of these areas jointly under the heading of study management. It includes protocol development, forms design, data management, quality control of laboratories and of field work in general, training field workers, monitoring the quality of field work, and relationships with study committees and study 257
258
JAMES E. GRIZZLE
centers. In general, it encompasses all of those activities thought to be necessary for the collection of data that are of adequate quality for the purposes of a study. Obviously, statisticians will not bear the full brunt of all these obligations, but they should be among those most concerned about taking the appropriate steps to meet a study’s goals for data quality. Statisticians who are specialists in sampling and their collaborators have designed special studies in several areas to evaluate the contributions to the mean square error made by interviewer bias, respondent bias, sampling error and various aspects of field work. These studies have provided sound criteria which can be used to allocate the resources available in the most efficient way for some types of surveys. In contrast, one seldom sees detailed exposition, supported by objective evidence, on the most efficient way to organize and manage clinical trials. Greenberg[3] has given an outline of some of the ingredients necessary for a successful study but he does not go into the cost aspects of study management. At the present time, methods are adopted or discarded on the basis of intuition. This author has never seen an objective evaluation of the extent to which any of the study components thought to be necessary actually contributes to the mean square error or to any other measure of the statistical precision or validity of a study. A number of plausible explanations exist for the lack of systematic assessment of the components that are necessary for a successful clinical intervention study in chronic disease. These studies are a relatively recent development. Not all centers of education in the relevant disciplines have faculties with first hand experience in the design and conduct of such studies. Most of the knowledge about design and maintenance of long term studies is transmitted by on-the-job training and other informal methods rather than by the usual academic routes. Until recent moves by coordinating centers for several NHLBI financed studies to hold annual meetings, there was very little exchange of technology among different coordinating units. One has the impression that the wheel is continually being reinvented but never evaluated. Probably the greatest obstacle is that the methods of organizing and maintaining an intervention study of high quality have not been recognized as a respectable area of inquiry. This in turn results in a lack of financial support for investigation into how the methods now in use can be improved and resource allocation made more rational. Without systematic investigations there are no journal articles or books, which are necessary to stimulate further research and facilitate a more orderly transfer of knowledge. Consequently, the academic community has not granted this area a high level of respectability. Apparently it falls in the interface of management, epidemiology, and statistics. Cochran [4] mentions that persons who are successful in key directive roles in long term studies are different from the ‘ivory tower’ concept of scientist.
“He must be able to assemble a team of workers and to maintain good relations among them: he must obtain the cooperation of various administrators and their agencies and must handle a considerable amount of paper work. Scientific competence alone does not guarantee success in this type of research: some scientists are too shy. and others too quarrelsome, to meet the requirements.”
The Case for Management
Research
for Large
Field Trials
259
Obviously all the desired personality attributes cannot be taught, but this should not prohibit the systematic development of a useful body of knowledge. Most, if not all, of the ingredients necessary for the development of design and management of studies of chronic disease as an area of scientific study and academic training are identifiable. The first requirement is the recognition that there is a body of knowledge to be acquired and that it is capable of organization and further development. Persons who are successful in this area should be recognized as productive scientists and treated as such in the scientific community. This should include publication, in scientific journals, of their research into the methodology of management of clinical trials in chronic disease, and recognition of the value of this work in the academic community when appointments and promotions are made. Systematic development of the field cannot take place without some financial support. Centers which have contracts or grants to coordinate studies of chronic disease should be provided with some funds for investigation into the methodologies they use. Experimental comparison of different methods now in use to accomplish the same ends would be a good starting place. A budget of even l-2”,, of the total money being spent in intervention studies in chronic diseases should lead to the development of useful knowledge. Without this knowledge scientists in this field cannot objectively allocate their resources in optimum ways to control cost. The standards for study design and administration appear to be increasing as a consequence of growing sophistication among clinical practitioners, epidemiologists, and statisticians and of concern for patient’s welfare. It is beginning to be recognized that costs increase with increasing rigor of study design and the ensuing problem of maintaining the design quality over an extended period of time. These costs may grow to the extent that the desired levels of rigor cannot be achieved for many important studies unless data are accumulated to evaluate the cost benefits of various measures undertaken to maintain the study. It is time to recognize this problem and to develop some reliable recommendations for use in the field.
REFERENCES Marmot M. Winklestein W, Jr: Epidemiologic observations on intervention trials for prevention of coronary heart disease. Am J Epid 101 : 177-181, 1975 Sondlk E, Brown B. Jr. Sdvers A: High Irisk subjects and the cost of large field trials. J C’hron Dis 27: 177- 18X. 1974 Greenberg B: Conduct of cooperative field and clinical trials. Am Stat 13: 13- 17, 1959 Cochran W: Research techniques in the study of human beings. Milbank Mem Fund Quart 33: 131-136. 1955
