Objective Criteria for Evaluating Occupational Health Programs SAMUEL B. WEBB, Jr., DrPH
A scoring system for evaluating occupational health programs is proposed.
Introduction Any program evaluation should be comprised of measurement criteria. These criteria may be purely subjective: a program may be evaluated in terms of what the people responsible for it believe it has accomplished. Other criteria may be objective: the program in question may be evaluated in terms of a model program encompassing certain standards. Evaluative criteria may also be composed of a combination of subjective and objective factors. Whenever possible program evaluation should be based on objective criteria. Such evaluation minimizes the hazards of subjective interpretation which can often lead to extensive disagreement and thereby detract from the meaningfulness and usefulness of the evaluation attempt. However, before the evaluation of a program can be discussed seriously, some agreement is needed about the objectives of "a program and its component parts."'
Program Definition Due to a lack of common agreement on the nature of occupational health programs, the task of evaluating these programs is immense. It can be argued that there can be no ideal occupational health program because of the diversified nature of American industry. Industry's varying size, geographic location, and production of diverse commodities and the resultant differing needs of employees tend to make an ideal program impractical. Nevertheless, certain basic minimal services should be rendered in any occupaDr. Webb is Associate Professor of Public Health and Deputy Director, Program in Hospital Administration, Yale University School of Medicine, New Haven, Connecticut 06510.
tional health program. Furthermore, it is these basic services which should distinguish an occupational health program from a mere first aid program or one that is designed only to assist personnel departments in selecting the most medically qualified applicants for employment. These basic services should not present such high standards that it would be unrealistic to assume that they could be achieved within present resource capabilities. Such services should, however, provide a basic standard of excellence. When this standard is achieved, new and higher standards should be set. Therefore, the pursuit of excellence in occupational health programs is a dynamic rather than a static process. Guidelines for occupational health programs have been promulgated by the Council on Occupational Health of the American Medical Association.2 Likewise, guidelines for evaluating occupational health programs designed to serve 2,500 to 25,000 individuals have been set forth by the Occupational Health Institute.3 In addition, components of an occupational health program have been outlined in some detail by Felton.4 All of these are commendable. If the program content outlined by Felton and the AMA and the guidelines proposed by the Occupational Health Institute were adhered to vigorously, certainly a state of excellence would be approached in providing occupational health services to employees. However, even if this were to happen, there is no existing method by which these programs could be evaluated through truly objective criteria. For example, the Occupational Health Institute states that one of the minimal standards for an occupational health program shall be the performance of a preplacement evaluation, but it does not say what the content of this evaluation should be. On the other hand, Felton concisely outlines the content of a preplacement health evaluation but does not indicate the relative importance of such an evaluation's several components. EVALUATING OCCUPATIONAL HEALTH PROGRAMS
31
Doran has proposed an occupational health medical audit program, which is comprehensive in scope, objectives, and content, but makes no attempt to relate what exists to what should be in an objective fashion.5 Accordingly, to undertake meaningful evaluation of occupational health programs a model occupational health program, comprised of basic minimal services, must be devised. Moreover, the components of such a program must be quantified, thereby providing a simple and flexible method for objectively evaluating a program's capability to achieve basic program goals. Therefore, the purpose of this paper is to develop an objective scoring system which will enable individual occupational health programs to evaluate their capabilities objectively in relation to a realistic, albeit theoretical, model occupational health program.
health program can be grouped according to a company's commitment toward the short range or long range health interests of its employees. Short range health commitments (Table 1) are defined as those occupational health services which are primarily economically motivated in that they will have an immediate payoff to the company in terms of an individual's functional and emotional fitness to perform an assigned task; these services thus are economically viable from a profit and loss standpoint (through reduction of Component 1
32
AJPH JANUARY, 1975, Vol. 65, No. 1
Component 3
-* Component 4
A Model Occupational Health Program To achieve acceptance, a theoretical model of an occupational health program, comprised of basic minimal services, should have a realistic base. Accordingly, the following model is based primarily upon published professional opinion. Liberal use has been made of material sponsored by the Council on Occupational Health of the American Medical Association2 and the Occupational Health Institute,3 as well as other published material in the field of occupational health.4 As a basic minimum, any occupational health program should be comprised of the following four component parts: * A written guiding philosophy and medical policy * A defined organizational structure * Adequate resources (physical facilities and staff) * A basic health service program. Furthermore, these component parts should be interrelated to the degree that they are mutually dependent on one another. They cannot be conceived of as isolated entities. Rather, they should form a symbiotic relationship in order for a program to be effective. A graphic presentation of this model is given in Figure 1. Central to this model are the specific goals of the program which should serve as a guide for program growth and development. Unless the goals are stated explicitly and can be documented and referred to easily too often a program's efforts will be nondirected or in conflict and progress will be difficult, if not impossible, to evaluate in terms of their achievement. The following basic goals are proposed: (1) to keep an individual healthy so that he can perform his assigned job with the maximal amount of efficiency; (2) to match an individual's capacity to perform a specified job with the physical and psychological demands inherent in that job. Goal 1 requires an occupational health service to be engaged in more functional activities than performing health evaluations and periodically monitoring the health status of employees in hazardous occupations or in age groups with a known above average specific disease incidence. These additional activities of an occupational
Component 2
Occupational Health Services
Short Range Health Connmitments
Long Range Health Commitments
Employee Health
FIGURE 1 A model occupational health program: the four basic components and their relationships to the production of employee health. TABLE 1-Occupational Health Services by Type of Commitment to Employee Health
Short Range Preplacement evaluations Preplacement laboratory procedures Personal protective devices as indicated by work environment Periodic occupational health evaluations Job transfer evaluations Job termination health evaluations Treatment for occupational illness or injury Emergency treatment for nonoccupational illness or injury Regular inspection of premises for potential hazards Long Range Regular health evaluations Health counseling Health education program Special surveys for case finding Retiree health program Alcohol control program Rehabilitation program Liaison with personal family physicians Liaison with community health insurance programs Advice to management about health insurance programs
absenteeism and employee turnover, increasing job performance and efficiency, reduction of insurance premiums for workmen's compensation, etc.). To be sure, there may be immediate health benefits to the employee, but this is a tangential rather than deliberate benefit as it is predicated on an employee remaining in the company's employ, for without his services company profits might be jeopardized. In contrast, long range health commitments (Table 1) are defined as those services which are motivated by a concem for health, not economics. They are designed for the purpose of installing good health practices in the employee community and promoting an employee's health for the employee's, rather than the company's, benefit. These services, rather than short range health commitments, will have the most impact on achieving the goal of keeping an individual healthy. They recognize that an employee's health status is related to extra-employment factors, such as the employee's family relationships, his personal living conditions, and his basic health habits. In contrast to the short range service commitments, these long range commitments probably are not economical in the short run. In addition, it is difficult to obtain an accurate assessment of any tangential economic benefit in terms of a profit and loss statement. These service commitments embody the concepts of primary (health counseling and health education), secondary (regular health evaluations and special surveys), and tertiary prevention (rehabilitation). They recognize that health implies physical, social, and mental well-being. They also recognize that to achieve health, efforts must be geared toward man's broadest ecological relationships rather than isolated in the restrictive milieu of the work environment. Besides these specific activities, to keep an employee healthy means that a continuum of care must be provided. The occupational physician cannot prescribe treatments which are contraindicated in view of a treatment prescribed by the employee's personal physician and vice versa. Likewise, neither the occupational nor private family physician can provide all of the resources which technology has made available to treat illness. Therefore, an occupational health program must make provisions for formal liaison with the employee's personal physician and with existing community medical care resources. In addtion, management should receive the advice of the occupational health director about company programs which have a bearing on employee health. At a minimum, this should include the company health insurance program. The Scoring System and Program Components The proposed evaluation scoring system is intended to be a simple and flexible method for objectively evaluating an occupational health program in terms of: (1) its capability to fulfill its mission; (2) its commitment to the long range health interests of its employees; and (3) its commitment to the short range health interests of its employees, in order to assess a company's capability to achieve basic program goals. In addition, this system can be
used to evaluate whether different types of programs really have an influence on such items as employee hospital utilization and employee health status in general. The quantification of the proposed four occupational health program components into a net "score" for comparing varying types of programs is difficult. However, it can be attempted by assigning a weight to each component of the proposed model and by scaling the values within these weightings from low to high. This scheme has been used successfully in another instance, and in the author's opinion the general approach is valid.6 In any scoring system, the weights assigned to different components are somewhat arbitrary. However, they may be chosen with reasonable judgment on the degree to which each individual component contributes to the overall goals of the program. Accordingly, the proposed scoring system for the four components of the program is as follows:
Program Component
Weighting
1. Guiding philosophy and policy 2. Organizational structure 3. Resources 4. Health services program Maximal score
10 10 20 60 100
The assigned weights are based on the importance of the contributions of the individual components toward fulfilling the basic goals of the model occupational health program. It is evident that the total score of a program is influenced greatly by the subscores of components 3 and 4, particularly component 4. This component, however, has the most significant impact in terms of employee health and, as stated earlier, is the most tangible indication of a company's commitment to the health of its employees. While a company may have the best intentions in the world, state their philosophy and policy implicitly to support these intentions, and devise an ideal organizational structure within which a program should be able to function expeditiously, without a strong program of health services, and without the resources necessary to implement such a program, good intentions-in terms of philosophy, policy, and organizational structure-will not produce significant results in terms of employee health. Accordingly, the resource component, defined as facilities and staff, is given a higher weight than the components of philosophy and organization. However, while resources are important, a company could have resources and still not provide important health services, thereby having a minimal effect on employee health. Therefore, the weighting for the resource component (No. 3) is significantly lower than the important health services program component (No. 4), which is judged to be the major determinant affecting employee health. Upon the basis of the model, assigned weights are given to individual items which, when added together cumulatively, comprise the component score. These individual weights are related to the total component weight and are EVALUATING OCCUPATIONAL HEALTH PROGRAMS
33
based on the importance that the individual component item has in fulfilling the two basic goals of an occupational health program. Therefore, component items which are related directly to a company's long range commitment to the health of its employees and therefore geared toward fulfilling goal 1-keeping an individual healthy-are given significantly higher weights. For each component, the individual item weights are presented in Appendix A. Component 3-resources-is in one way the most subjective and arbitrary, and in another, the most objective weighting of the several components. In terms of facilities, it may be subjective and arbitrary, as one observer might feel that although facilities are available for the performance of health evaluations and counseling, they are inadequate for the actual performance of these functions due to failures of design or material. Similarly, the interpretation of adequacy is subjective as are the terms "attractive" and "well kept." To compensate for this weakness, the maximal score for this part of component 3 is 4. This is low enough so as not to influence the total program score (a maximum of 100) too significantly. In addition, to offset this weakness, part B of component 3-staff-is objective and carries a significantly higher maximal score of 16. However, the method of calculating the number of points to be given for this part of component 3 needs clarification and explanation. It is difficult to assign any particular weight to this part of component 3 as staff size must vary in accordance with a company's size, location, and mission. To circumvent this problem, a minimal staff component is proposed, based solely on the number of employees in a company. This is flexible and can therefore be calculated realistically by any company. In addition, companies can calculate what their staff component should be in terms of a basic standard and compare this to their actual available staff. In this way, a cornpany can immediately recognize any staffing deficiency. The minimal staff component proposed is based on the following formula: Physician-hours required = Number of physicians X: (a) 2 hr per week for the first 100 employees; (b) 4 hr per week for each additional 100 employees up to 500 employees; (c) 2 hr per week for each additional 100 employees. In this way, minimal physician staff can be calculated for individual companies. However, it has been shown that it is the combined team approach of full-time occupational health nurses and physicians which determines the level of preventive health activities in a program.7 Accordingly, staff should be judged in terms of the concept of a full-time health team. There is no attempt at equating a physician's time to a nurse's time in terms of some theoretical proportion. These individuals perform sufficiently different functions so that any attempt to develop theoretical proportionate net worth (e.g., how many nurse-hours equal a physician's hour of time, or vice versa) is unrealistic. Each professional brings a discrete body of knowledge and expertise to the employee. Their functions and activities complement one another. As such, they should not be compared in terms of proportionate net worth. 34
AJPH JANUARY, 1975, Vol. 65, No. 1
The number of professional health team man-hours per day per 100 employees is a valid measurement of the component of staff. The number of physician-hours worked per day, or week, is based on a 40-hr week, which is customary in industry. However, such a figure may be conservative in terms of physician man-hours per week. A recent study shows that in terms of the national average, physicians practiced 55.8 hr per week. In terns of specialty, for direct care of patients, the range is 30.3 hr per week (radiologists) to 49.2 hr per week (general practitioners). The average is 45.3 hr per week. Interestingly, physicians in non-solo practice have a higher number of man-hours devoted to direct patient care (47.5 hr per week) than physicians in solo practice (44.0 hr per week).8 Therefore, it is probable that physicians specializing in occupational health may, in fact, devote more than 40 hr per week to their patients. On the other hand, it is unusual for nurses to work more than a 40-hr week, whether in office, hospital, or public health nursing. Thus, this figure is probably fairly accurate in terms of occupational health nurse-hours. Professional health team man-hours are calculated in terns of the number of 100-employee cohorts. Such a calculation is realistic and professionally acceptable. In order to ascertain the staff score, the following calculations are necessary:
PHYSICIANS 1. Number of physicians X 8 hr per day = Number of physician-hours per day. 2. Number of employees . 100 = Number of employee cohorts of 100. 3. Number of physician-hours per day . Number of 100-employee cohorts = Number of physician-hours per day available per 100 employees.
NURSES 4. Number of nurses X 8 hr per day = Number of nurse-hours per day. 5. Number of employees -. 100 = Number of employee cohorts of 100. 6. Number of nurse-hours per day * Number of 100-employee cohorts = Number of nurse-hours per day per 100 employees.
PROFESSIONAL HEALTH TEAM 7. Number of physician-hours per day per 100 employees + Number of nurse-hours per day per 100 employees = Number of professional health team manhours per day per 100 employees. In summary, No. 3 + No. 6 = No. 7 (Number of professional health team man-hours per day per 100
employees). These calculations are performed in terms of the proposed ideal minimal staff component and then the actual staff component. The actual staff component is then divided by the ideal staff component and multiplied by 16,
the maximal score. The resultant flgure is the number of points to be awarded. It should be noted that part-time physicians and nurses are not included in the basic staff score. An attempt to convert part-time professional personnel into full-time equivalents, when they are used solely on a referral basis, is too imprecise and probably academic. More importantly, however, any occupational health program should have full-time staff. It is conceivable that an individual occupational health program's staff could exceed the basic minimum which is proposed. In this event, it could be argued that the program should be awarded more than the maximum score of 16 points which this scheme allows. In other words, allowance should be made for bonus points as a program with more than a minimal staff should, theoretically, be able to devote more time to important preventive activities; institute additional services such as home care for the injured or ill employee, a visiting nurse service, and additional special surveys; or undertake intensive medical research programs in occupational health. Perhaps this is true. Maybe such a program would exert a stronger influence on employee health than the program which merely meets the minimal proposed standard. However, any such refinement in the proposed scoring system should await exploratory testing.
Summary An objective scoring system is proposed as a single and flexible method of evaluating occupational health programs to appraise the extent to which existing programs approach a basic standard of excellence as defined by a theoretical model based on published and professionally acceptable guidelines and standards. This proposed system emphasizes the importance of an interdependent relationship between four program components: (1) guiding philosophy and policy; (2) organizational structure; (3) resources; (4) occupational health services, and it stresses the importance of long range health commitments to employee health status in contrast to short range health commitments aimed primarily at an economic payoff to a sponsoring agency, institution, or company. The proposed evaluation scheme should enable self-evaluation by individual programs. Additionally, programs can utilize this evaluative tool to examine their influence on such important questions as employee hospital utilization and other specific elements of employee health in an objective, relatively simple manner.
ACKNOWLEDGMENT Grateful appreciation is expressed to Jean Spencer Felton, MD, Medical Director, Occupational Health Services, The County of Los Angeles, California, for help in the preparation of this paper.
References 1. Deniston, 0. L., and Rosenstock, I. M. Evaluating Health Programs. Public Health Rep. 85:835-840, 1970.
2. Council on Occupational Health. Scope, Objectives, and Functions of Occupational Health Programs. J. A. M. A. 175:533-536, 1960. 3. McKiever, M. F., and Siegel, G. S. Occupational Health Services for Employees. A Guide for State and Local Governments. PHS Publication No. 1041. U.S. Department of Heaith, Education, and Welfare, Public Health Service. U.S. Government Printing Office, Washington, DC, 1963. 4. Felton, J. S. The Organization and Operation of an Occupational Health Program. J. Occup. Med. 6:27-67, 93-103, 132-151, 1964. 5. Doran, W. T., Jr. The Medical Audit: A Tool in Appraising Occupational Health Programs. J. Occup. Med. 6:221-226, 1964. 6. Roemer, M. I., and Friedman, J. W. Medical Staff Organization in Hospitals: A New Typology. Hosp. Management 105: No. 4, 58-61, No. 5, 41-44, No. 6, 56-59, 1968. 7. Employee Health Services: A Study of Managerial Attitudes and Evaluations, conducted for the Public Health Service by the Institute for Social Research. University of Michigan Press, Ann Arbor, 1955. 8. Theodore, C. N., and Sutter, G. E. A Report on the First Periodic Survey of Physicians. J. A. M. A. 202:
516-524, 1967.
APPENDIX A A MODEL OCCUPATIONAL HEALTH PROGRAM AND SCORING SYSTEM
Component 1: A Written Guiding Philosophy and Policy
Weighting
A. Goals 1. To keep an individual healthy so that he can perform his assigned job with the maximal amount of efficiency. 2. To match an individual's capacity to perform a specified job with the physical and psychological demands inherent in that job. B. Objectives 1. The maintenance of the health of the worker through specific preventive medicine procedures, frequent check on employee health status, health education activities, and detection and control of potential health hazards inherent in the work environment. 2. The provision of emergency medical care for occupational and nonoccupational illnesses or injuries. 3. The provision of definitive care and rehabilitation for occupational illnesses or injuries in keeping with the medical or surgical skills of the staff. C. Medical policy 1. Preplacement health evaluations for all applicants. 2. Periodic health evaluations and/or health surveys of all employees or employees in
1.5
0.5
EVALUATING OCCUPATIONAL HEALTH PROGRAMS
1.0
0.5 0.5
0.35 0.75
35
Weigh ting
3. 4.
5. 6. 7.
certain age groups regardless of type of work. Special health evaluations for employees with specific disabilities and new employees transferring to different jobs. Employee health counseling and health promotion activities. Treatment of the alcoholic. Rehabilitation of the ill or injured employee. Formal liaison with personal family
0.35 0.75 0.50 0.50 0.50
physicians.
8. Formal liaison with community medical care resources. 9. Termination health evaluations. 10. Medical care is rendered by licensed, qualified professionals familiar with the work environment. 11. Professional consultant services are used for examinations and definitive care beyond the expertise of the full-time staff. 12. A record system is maintained that: a. Is confidential b. Contains job descriptions c. Is reviewed at regular intervals for updating d. Is used for purposes of medical investigation Maximal Score, Component 1:
0.50 0.35 0.50
0.50
0.15 0.15 0.10 0.05
10.00
Component 2: Organizational Structure 1. Health director reports directly to top management: a. President b. Vice President c. Department or Section Head 2. Top management is familiar with managerial interpretation of medical policy and assistance can be assured. 3. Health director is involved directly in advising management regarding company policy on: a. Employee health insurance benefits b. Retirement program Medical department budget and proc. gram direction d. Industrial relations criteria of efficiency, absenteeism, or labor turnover and effective work performance 4. Formal liaison with company committee concerned with: a. Safety b. Disaster control Health c. Maximal Score, Component 2: 36
AJPH JANUARY, 1975, Vol. 65, No. 1
3.0 2.0 0.5 1.0
1.0 0.5 1.5
Component 3: Resources
Weighting
1. Facilities a. Facilities are available for the perfornance of health evaluations and counseling. b. Facilities are adequate for dispensing emergency care for occupational and nonoccupational illnesses or injuries. c. Facilities are attractive, well kept. 2. Staff (Actual number of professional health team man-hours per 100 employees per day - Ideal) X Maximum points Maximal Score, Component 3:
1.5 1.5 1.0
16.0 20.0
Component 4: Occupational Health Services Program 1. Preplacement health evaluation a. Personal history (0.3) b. Family history (0.3) c. Occupational history (jobs held, dates, and places) (0.3) d. Type of work exposure (0.3) e. Organ inventory (0.3) f. Functional capacity (0.75) g. Psychological evaluation (0.75) 2. Preplacement laboratory procedures a. Chest radiography (0.3) b. Audiometry (0.3) c. Visual testing (0.3) d. Electrocardiography for males over 50 (0.3) e. Hematological studies (0.3) f. Urinalysis (0.3) 1. Chemical (0.15) 2. Microscopic (0.15) g. Serodiagnostic test for syphilis (0.3) h. Blood typing and Rh factor determination (0.3) i. Other radiographic, hematological, or laboratory studies as indicated by previous history, former job-related injury, or illness, or by examination
1.0
findings (0.3) 3. Classification system for fitness for employment a. Physically fit for any job (0.4) b. Physically fit for any job, but has minor remediable defects (0.4) c. Physically fit for modified work only in accordance with noted restrictions
0.5 0.5 1.0 10.0
(0.4) d. Physically unqualified for any job applied for (0.4) e. Temporarily deferred (0.4)
3.0
3.0
2.0
Weighting
4. Immunization program 5. Personal protective devices as indicated by work environment 6. Regular health evaluation a. All employees (3.0) b. Management (0.25) c. Employees in certain age groups (1.0) d. Employees in specific disabilities (0.75) 7. Periodic occupational health evaluation a. Employees exposed to hazards (1.0) b. Employees in certain age groups
(0.8) c. Other illness or absence (0.2) 8. Job transfer evaluation a. All employees (2.0) b. Only some employees (0.5) 9. Specific disability examinations at appropriate intervals for those hired with impairments 10. Termination physical evaluation a. All employees (1.0)
1.0 3.0
Weighting
11.
5.0 12. 13. 2.0
14.
b. Only some employees (0.2) Health counseling a. Does not include psychological (-2.0) Special surveys for case finding, according to appropriate age and sex groups and geographic locations Health education program a. Posters, flyers (0.25) b. Small group discussion (3.0) c. Discussion with supervisors (1.0) d. Community seminars (0.75) Treatment for occupational illness or
5.0 5.0 5.0
3.0
injury 2.0 2.0 1.0
15. Emergency treatment for nonoccupational illness or injury 16. Retiree health program 17. Regular inspection of plant for potential hazards 18. Alcohol control program 19. Rehabilitation program Maximal Score, Component 4:
MULTIDISCIPLINARY CONFERENCE ON HEALTH CARE ETHICS A "Conversation in the Disciplines" entitled "Ethical Dilemmas and Health Care Delivery: A Multidisciplinary Approach to Issues and Decision-Making" will be held April 25-26, 1975, at the State University of New York at Buffalo. Faculty of health sciences and related disciplines are invited to participate. For further information, write: Mila A. Aroskar, Associate Professor, School of Nursing, SUNYAB, Buffalo, NY 14214.
1975 ENVIRONMENTAL HEALTH CONFERENCE AND EXPOSITION The 1975 Environmental Health Conference and Exposition of the National Environmental Health Association will be held June 28 to July 3, 1975, at the Downtown Radisson, Minneapolis, MN. More than 1000 environmental health personnel concerned with food sanitation, waste disposal, air and water pollution, occupational safety and health, hospital and nursing home environments and other institutions, urban and rural housing, administration and planning, and education are expected to attend. Approximately 50 commercial exhibits are planned, along with educational materials and other environmental services. For more information concerning the meeting contact: the National Environmental Health Association, 1600 Pennsylvania, Denver, CO 80203.
3.0 2.0 3.0 5.0 5.0 60.0
A scoring system for evaluating occupational health programs is proposed.
Introduction Any program evaluation should be comprised of measurement criteria. These criteria may be purely subjective: a program may be evaluated in terms of what the people responsible for it believe it has accomplished. Other criteria may be objective: the program in question may be evaluated in terms of a model program encompassing certain standards. Evaluative criteria may also be composed of a combination of subjective and objective factors. Whenever possible program evaluation should be based on objective criteria. Such evaluation minimizes the hazards of subjective interpretation which can often lead to extensive disagreement and thereby detract from the meaningfulness and usefulness of the evaluation attempt. However, before the evaluation of a program can be discussed seriously, some agreement is needed about the objectives of "a program and its component parts."'
Program Definition Due to a lack of common agreement on the nature of occupational health programs, the task of evaluating these programs is immense. It can be argued that there can be no ideal occupational health program because of the diversified nature of American industry. Industry's varying size, geographic location, and production of diverse commodities and the resultant differing needs of employees tend to make an ideal program impractical. Nevertheless, certain basic minimal services should be rendered in any occupaDr. Webb is Associate Professor of Public Health and Deputy Director, Program in Hospital Administration, Yale University School of Medicine, New Haven, Connecticut 06510.
tional health program. Furthermore, it is these basic services which should distinguish an occupational health program from a mere first aid program or one that is designed only to assist personnel departments in selecting the most medically qualified applicants for employment. These basic services should not present such high standards that it would be unrealistic to assume that they could be achieved within present resource capabilities. Such services should, however, provide a basic standard of excellence. When this standard is achieved, new and higher standards should be set. Therefore, the pursuit of excellence in occupational health programs is a dynamic rather than a static process. Guidelines for occupational health programs have been promulgated by the Council on Occupational Health of the American Medical Association.2 Likewise, guidelines for evaluating occupational health programs designed to serve 2,500 to 25,000 individuals have been set forth by the Occupational Health Institute.3 In addition, components of an occupational health program have been outlined in some detail by Felton.4 All of these are commendable. If the program content outlined by Felton and the AMA and the guidelines proposed by the Occupational Health Institute were adhered to vigorously, certainly a state of excellence would be approached in providing occupational health services to employees. However, even if this were to happen, there is no existing method by which these programs could be evaluated through truly objective criteria. For example, the Occupational Health Institute states that one of the minimal standards for an occupational health program shall be the performance of a preplacement evaluation, but it does not say what the content of this evaluation should be. On the other hand, Felton concisely outlines the content of a preplacement health evaluation but does not indicate the relative importance of such an evaluation's several components. EVALUATING OCCUPATIONAL HEALTH PROGRAMS
31
Doran has proposed an occupational health medical audit program, which is comprehensive in scope, objectives, and content, but makes no attempt to relate what exists to what should be in an objective fashion.5 Accordingly, to undertake meaningful evaluation of occupational health programs a model occupational health program, comprised of basic minimal services, must be devised. Moreover, the components of such a program must be quantified, thereby providing a simple and flexible method for objectively evaluating a program's capability to achieve basic program goals. Therefore, the purpose of this paper is to develop an objective scoring system which will enable individual occupational health programs to evaluate their capabilities objectively in relation to a realistic, albeit theoretical, model occupational health program.
health program can be grouped according to a company's commitment toward the short range or long range health interests of its employees. Short range health commitments (Table 1) are defined as those occupational health services which are primarily economically motivated in that they will have an immediate payoff to the company in terms of an individual's functional and emotional fitness to perform an assigned task; these services thus are economically viable from a profit and loss standpoint (through reduction of Component 1
32
AJPH JANUARY, 1975, Vol. 65, No. 1
Component 3
-* Component 4
A Model Occupational Health Program To achieve acceptance, a theoretical model of an occupational health program, comprised of basic minimal services, should have a realistic base. Accordingly, the following model is based primarily upon published professional opinion. Liberal use has been made of material sponsored by the Council on Occupational Health of the American Medical Association2 and the Occupational Health Institute,3 as well as other published material in the field of occupational health.4 As a basic minimum, any occupational health program should be comprised of the following four component parts: * A written guiding philosophy and medical policy * A defined organizational structure * Adequate resources (physical facilities and staff) * A basic health service program. Furthermore, these component parts should be interrelated to the degree that they are mutually dependent on one another. They cannot be conceived of as isolated entities. Rather, they should form a symbiotic relationship in order for a program to be effective. A graphic presentation of this model is given in Figure 1. Central to this model are the specific goals of the program which should serve as a guide for program growth and development. Unless the goals are stated explicitly and can be documented and referred to easily too often a program's efforts will be nondirected or in conflict and progress will be difficult, if not impossible, to evaluate in terms of their achievement. The following basic goals are proposed: (1) to keep an individual healthy so that he can perform his assigned job with the maximal amount of efficiency; (2) to match an individual's capacity to perform a specified job with the physical and psychological demands inherent in that job. Goal 1 requires an occupational health service to be engaged in more functional activities than performing health evaluations and periodically monitoring the health status of employees in hazardous occupations or in age groups with a known above average specific disease incidence. These additional activities of an occupational
Component 2
Occupational Health Services
Short Range Health Connmitments
Long Range Health Commitments
Employee Health
FIGURE 1 A model occupational health program: the four basic components and their relationships to the production of employee health. TABLE 1-Occupational Health Services by Type of Commitment to Employee Health
Short Range Preplacement evaluations Preplacement laboratory procedures Personal protective devices as indicated by work environment Periodic occupational health evaluations Job transfer evaluations Job termination health evaluations Treatment for occupational illness or injury Emergency treatment for nonoccupational illness or injury Regular inspection of premises for potential hazards Long Range Regular health evaluations Health counseling Health education program Special surveys for case finding Retiree health program Alcohol control program Rehabilitation program Liaison with personal family physicians Liaison with community health insurance programs Advice to management about health insurance programs
absenteeism and employee turnover, increasing job performance and efficiency, reduction of insurance premiums for workmen's compensation, etc.). To be sure, there may be immediate health benefits to the employee, but this is a tangential rather than deliberate benefit as it is predicated on an employee remaining in the company's employ, for without his services company profits might be jeopardized. In contrast, long range health commitments (Table 1) are defined as those services which are motivated by a concem for health, not economics. They are designed for the purpose of installing good health practices in the employee community and promoting an employee's health for the employee's, rather than the company's, benefit. These services, rather than short range health commitments, will have the most impact on achieving the goal of keeping an individual healthy. They recognize that an employee's health status is related to extra-employment factors, such as the employee's family relationships, his personal living conditions, and his basic health habits. In contrast to the short range service commitments, these long range commitments probably are not economical in the short run. In addition, it is difficult to obtain an accurate assessment of any tangential economic benefit in terms of a profit and loss statement. These service commitments embody the concepts of primary (health counseling and health education), secondary (regular health evaluations and special surveys), and tertiary prevention (rehabilitation). They recognize that health implies physical, social, and mental well-being. They also recognize that to achieve health, efforts must be geared toward man's broadest ecological relationships rather than isolated in the restrictive milieu of the work environment. Besides these specific activities, to keep an employee healthy means that a continuum of care must be provided. The occupational physician cannot prescribe treatments which are contraindicated in view of a treatment prescribed by the employee's personal physician and vice versa. Likewise, neither the occupational nor private family physician can provide all of the resources which technology has made available to treat illness. Therefore, an occupational health program must make provisions for formal liaison with the employee's personal physician and with existing community medical care resources. In addtion, management should receive the advice of the occupational health director about company programs which have a bearing on employee health. At a minimum, this should include the company health insurance program. The Scoring System and Program Components The proposed evaluation scoring system is intended to be a simple and flexible method for objectively evaluating an occupational health program in terms of: (1) its capability to fulfill its mission; (2) its commitment to the long range health interests of its employees; and (3) its commitment to the short range health interests of its employees, in order to assess a company's capability to achieve basic program goals. In addition, this system can be
used to evaluate whether different types of programs really have an influence on such items as employee hospital utilization and employee health status in general. The quantification of the proposed four occupational health program components into a net "score" for comparing varying types of programs is difficult. However, it can be attempted by assigning a weight to each component of the proposed model and by scaling the values within these weightings from low to high. This scheme has been used successfully in another instance, and in the author's opinion the general approach is valid.6 In any scoring system, the weights assigned to different components are somewhat arbitrary. However, they may be chosen with reasonable judgment on the degree to which each individual component contributes to the overall goals of the program. Accordingly, the proposed scoring system for the four components of the program is as follows:
Program Component
Weighting
1. Guiding philosophy and policy 2. Organizational structure 3. Resources 4. Health services program Maximal score
10 10 20 60 100
The assigned weights are based on the importance of the contributions of the individual components toward fulfilling the basic goals of the model occupational health program. It is evident that the total score of a program is influenced greatly by the subscores of components 3 and 4, particularly component 4. This component, however, has the most significant impact in terms of employee health and, as stated earlier, is the most tangible indication of a company's commitment to the health of its employees. While a company may have the best intentions in the world, state their philosophy and policy implicitly to support these intentions, and devise an ideal organizational structure within which a program should be able to function expeditiously, without a strong program of health services, and without the resources necessary to implement such a program, good intentions-in terms of philosophy, policy, and organizational structure-will not produce significant results in terms of employee health. Accordingly, the resource component, defined as facilities and staff, is given a higher weight than the components of philosophy and organization. However, while resources are important, a company could have resources and still not provide important health services, thereby having a minimal effect on employee health. Therefore, the weighting for the resource component (No. 3) is significantly lower than the important health services program component (No. 4), which is judged to be the major determinant affecting employee health. Upon the basis of the model, assigned weights are given to individual items which, when added together cumulatively, comprise the component score. These individual weights are related to the total component weight and are EVALUATING OCCUPATIONAL HEALTH PROGRAMS
33
based on the importance that the individual component item has in fulfilling the two basic goals of an occupational health program. Therefore, component items which are related directly to a company's long range commitment to the health of its employees and therefore geared toward fulfilling goal 1-keeping an individual healthy-are given significantly higher weights. For each component, the individual item weights are presented in Appendix A. Component 3-resources-is in one way the most subjective and arbitrary, and in another, the most objective weighting of the several components. In terms of facilities, it may be subjective and arbitrary, as one observer might feel that although facilities are available for the performance of health evaluations and counseling, they are inadequate for the actual performance of these functions due to failures of design or material. Similarly, the interpretation of adequacy is subjective as are the terms "attractive" and "well kept." To compensate for this weakness, the maximal score for this part of component 3 is 4. This is low enough so as not to influence the total program score (a maximum of 100) too significantly. In addition, to offset this weakness, part B of component 3-staff-is objective and carries a significantly higher maximal score of 16. However, the method of calculating the number of points to be given for this part of component 3 needs clarification and explanation. It is difficult to assign any particular weight to this part of component 3 as staff size must vary in accordance with a company's size, location, and mission. To circumvent this problem, a minimal staff component is proposed, based solely on the number of employees in a company. This is flexible and can therefore be calculated realistically by any company. In addition, companies can calculate what their staff component should be in terms of a basic standard and compare this to their actual available staff. In this way, a cornpany can immediately recognize any staffing deficiency. The minimal staff component proposed is based on the following formula: Physician-hours required = Number of physicians X: (a) 2 hr per week for the first 100 employees; (b) 4 hr per week for each additional 100 employees up to 500 employees; (c) 2 hr per week for each additional 100 employees. In this way, minimal physician staff can be calculated for individual companies. However, it has been shown that it is the combined team approach of full-time occupational health nurses and physicians which determines the level of preventive health activities in a program.7 Accordingly, staff should be judged in terms of the concept of a full-time health team. There is no attempt at equating a physician's time to a nurse's time in terms of some theoretical proportion. These individuals perform sufficiently different functions so that any attempt to develop theoretical proportionate net worth (e.g., how many nurse-hours equal a physician's hour of time, or vice versa) is unrealistic. Each professional brings a discrete body of knowledge and expertise to the employee. Their functions and activities complement one another. As such, they should not be compared in terms of proportionate net worth. 34
AJPH JANUARY, 1975, Vol. 65, No. 1
The number of professional health team man-hours per day per 100 employees is a valid measurement of the component of staff. The number of physician-hours worked per day, or week, is based on a 40-hr week, which is customary in industry. However, such a figure may be conservative in terms of physician man-hours per week. A recent study shows that in terms of the national average, physicians practiced 55.8 hr per week. In terns of specialty, for direct care of patients, the range is 30.3 hr per week (radiologists) to 49.2 hr per week (general practitioners). The average is 45.3 hr per week. Interestingly, physicians in non-solo practice have a higher number of man-hours devoted to direct patient care (47.5 hr per week) than physicians in solo practice (44.0 hr per week).8 Therefore, it is probable that physicians specializing in occupational health may, in fact, devote more than 40 hr per week to their patients. On the other hand, it is unusual for nurses to work more than a 40-hr week, whether in office, hospital, or public health nursing. Thus, this figure is probably fairly accurate in terms of occupational health nurse-hours. Professional health team man-hours are calculated in terns of the number of 100-employee cohorts. Such a calculation is realistic and professionally acceptable. In order to ascertain the staff score, the following calculations are necessary:
PHYSICIANS 1. Number of physicians X 8 hr per day = Number of physician-hours per day. 2. Number of employees . 100 = Number of employee cohorts of 100. 3. Number of physician-hours per day . Number of 100-employee cohorts = Number of physician-hours per day available per 100 employees.
NURSES 4. Number of nurses X 8 hr per day = Number of nurse-hours per day. 5. Number of employees -. 100 = Number of employee cohorts of 100. 6. Number of nurse-hours per day * Number of 100-employee cohorts = Number of nurse-hours per day per 100 employees.
PROFESSIONAL HEALTH TEAM 7. Number of physician-hours per day per 100 employees + Number of nurse-hours per day per 100 employees = Number of professional health team manhours per day per 100 employees. In summary, No. 3 + No. 6 = No. 7 (Number of professional health team man-hours per day per 100
employees). These calculations are performed in terms of the proposed ideal minimal staff component and then the actual staff component. The actual staff component is then divided by the ideal staff component and multiplied by 16,
the maximal score. The resultant flgure is the number of points to be awarded. It should be noted that part-time physicians and nurses are not included in the basic staff score. An attempt to convert part-time professional personnel into full-time equivalents, when they are used solely on a referral basis, is too imprecise and probably academic. More importantly, however, any occupational health program should have full-time staff. It is conceivable that an individual occupational health program's staff could exceed the basic minimum which is proposed. In this event, it could be argued that the program should be awarded more than the maximum score of 16 points which this scheme allows. In other words, allowance should be made for bonus points as a program with more than a minimal staff should, theoretically, be able to devote more time to important preventive activities; institute additional services such as home care for the injured or ill employee, a visiting nurse service, and additional special surveys; or undertake intensive medical research programs in occupational health. Perhaps this is true. Maybe such a program would exert a stronger influence on employee health than the program which merely meets the minimal proposed standard. However, any such refinement in the proposed scoring system should await exploratory testing.
Summary An objective scoring system is proposed as a single and flexible method of evaluating occupational health programs to appraise the extent to which existing programs approach a basic standard of excellence as defined by a theoretical model based on published and professionally acceptable guidelines and standards. This proposed system emphasizes the importance of an interdependent relationship between four program components: (1) guiding philosophy and policy; (2) organizational structure; (3) resources; (4) occupational health services, and it stresses the importance of long range health commitments to employee health status in contrast to short range health commitments aimed primarily at an economic payoff to a sponsoring agency, institution, or company. The proposed evaluation scheme should enable self-evaluation by individual programs. Additionally, programs can utilize this evaluative tool to examine their influence on such important questions as employee hospital utilization and other specific elements of employee health in an objective, relatively simple manner.
ACKNOWLEDGMENT Grateful appreciation is expressed to Jean Spencer Felton, MD, Medical Director, Occupational Health Services, The County of Los Angeles, California, for help in the preparation of this paper.
References 1. Deniston, 0. L., and Rosenstock, I. M. Evaluating Health Programs. Public Health Rep. 85:835-840, 1970.
2. Council on Occupational Health. Scope, Objectives, and Functions of Occupational Health Programs. J. A. M. A. 175:533-536, 1960. 3. McKiever, M. F., and Siegel, G. S. Occupational Health Services for Employees. A Guide for State and Local Governments. PHS Publication No. 1041. U.S. Department of Heaith, Education, and Welfare, Public Health Service. U.S. Government Printing Office, Washington, DC, 1963. 4. Felton, J. S. The Organization and Operation of an Occupational Health Program. J. Occup. Med. 6:27-67, 93-103, 132-151, 1964. 5. Doran, W. T., Jr. The Medical Audit: A Tool in Appraising Occupational Health Programs. J. Occup. Med. 6:221-226, 1964. 6. Roemer, M. I., and Friedman, J. W. Medical Staff Organization in Hospitals: A New Typology. Hosp. Management 105: No. 4, 58-61, No. 5, 41-44, No. 6, 56-59, 1968. 7. Employee Health Services: A Study of Managerial Attitudes and Evaluations, conducted for the Public Health Service by the Institute for Social Research. University of Michigan Press, Ann Arbor, 1955. 8. Theodore, C. N., and Sutter, G. E. A Report on the First Periodic Survey of Physicians. J. A. M. A. 202:
516-524, 1967.
APPENDIX A A MODEL OCCUPATIONAL HEALTH PROGRAM AND SCORING SYSTEM
Component 1: A Written Guiding Philosophy and Policy
Weighting
A. Goals 1. To keep an individual healthy so that he can perform his assigned job with the maximal amount of efficiency. 2. To match an individual's capacity to perform a specified job with the physical and psychological demands inherent in that job. B. Objectives 1. The maintenance of the health of the worker through specific preventive medicine procedures, frequent check on employee health status, health education activities, and detection and control of potential health hazards inherent in the work environment. 2. The provision of emergency medical care for occupational and nonoccupational illnesses or injuries. 3. The provision of definitive care and rehabilitation for occupational illnesses or injuries in keeping with the medical or surgical skills of the staff. C. Medical policy 1. Preplacement health evaluations for all applicants. 2. Periodic health evaluations and/or health surveys of all employees or employees in
1.5
0.5
EVALUATING OCCUPATIONAL HEALTH PROGRAMS
1.0
0.5 0.5
0.35 0.75
35
Weigh ting
3. 4.
5. 6. 7.
certain age groups regardless of type of work. Special health evaluations for employees with specific disabilities and new employees transferring to different jobs. Employee health counseling and health promotion activities. Treatment of the alcoholic. Rehabilitation of the ill or injured employee. Formal liaison with personal family
0.35 0.75 0.50 0.50 0.50
physicians.
8. Formal liaison with community medical care resources. 9. Termination health evaluations. 10. Medical care is rendered by licensed, qualified professionals familiar with the work environment. 11. Professional consultant services are used for examinations and definitive care beyond the expertise of the full-time staff. 12. A record system is maintained that: a. Is confidential b. Contains job descriptions c. Is reviewed at regular intervals for updating d. Is used for purposes of medical investigation Maximal Score, Component 1:
0.50 0.35 0.50
0.50
0.15 0.15 0.10 0.05
10.00
Component 2: Organizational Structure 1. Health director reports directly to top management: a. President b. Vice President c. Department or Section Head 2. Top management is familiar with managerial interpretation of medical policy and assistance can be assured. 3. Health director is involved directly in advising management regarding company policy on: a. Employee health insurance benefits b. Retirement program Medical department budget and proc. gram direction d. Industrial relations criteria of efficiency, absenteeism, or labor turnover and effective work performance 4. Formal liaison with company committee concerned with: a. Safety b. Disaster control Health c. Maximal Score, Component 2: 36
AJPH JANUARY, 1975, Vol. 65, No. 1
3.0 2.0 0.5 1.0
1.0 0.5 1.5
Component 3: Resources
Weighting
1. Facilities a. Facilities are available for the perfornance of health evaluations and counseling. b. Facilities are adequate for dispensing emergency care for occupational and nonoccupational illnesses or injuries. c. Facilities are attractive, well kept. 2. Staff (Actual number of professional health team man-hours per 100 employees per day - Ideal) X Maximum points Maximal Score, Component 3:
1.5 1.5 1.0
16.0 20.0
Component 4: Occupational Health Services Program 1. Preplacement health evaluation a. Personal history (0.3) b. Family history (0.3) c. Occupational history (jobs held, dates, and places) (0.3) d. Type of work exposure (0.3) e. Organ inventory (0.3) f. Functional capacity (0.75) g. Psychological evaluation (0.75) 2. Preplacement laboratory procedures a. Chest radiography (0.3) b. Audiometry (0.3) c. Visual testing (0.3) d. Electrocardiography for males over 50 (0.3) e. Hematological studies (0.3) f. Urinalysis (0.3) 1. Chemical (0.15) 2. Microscopic (0.15) g. Serodiagnostic test for syphilis (0.3) h. Blood typing and Rh factor determination (0.3) i. Other radiographic, hematological, or laboratory studies as indicated by previous history, former job-related injury, or illness, or by examination
1.0
findings (0.3) 3. Classification system for fitness for employment a. Physically fit for any job (0.4) b. Physically fit for any job, but has minor remediable defects (0.4) c. Physically fit for modified work only in accordance with noted restrictions
0.5 0.5 1.0 10.0
(0.4) d. Physically unqualified for any job applied for (0.4) e. Temporarily deferred (0.4)
3.0
3.0
2.0
Weighting
4. Immunization program 5. Personal protective devices as indicated by work environment 6. Regular health evaluation a. All employees (3.0) b. Management (0.25) c. Employees in certain age groups (1.0) d. Employees in specific disabilities (0.75) 7. Periodic occupational health evaluation a. Employees exposed to hazards (1.0) b. Employees in certain age groups
(0.8) c. Other illness or absence (0.2) 8. Job transfer evaluation a. All employees (2.0) b. Only some employees (0.5) 9. Specific disability examinations at appropriate intervals for those hired with impairments 10. Termination physical evaluation a. All employees (1.0)
1.0 3.0
Weighting
11.
5.0 12. 13. 2.0
14.
b. Only some employees (0.2) Health counseling a. Does not include psychological (-2.0) Special surveys for case finding, according to appropriate age and sex groups and geographic locations Health education program a. Posters, flyers (0.25) b. Small group discussion (3.0) c. Discussion with supervisors (1.0) d. Community seminars (0.75) Treatment for occupational illness or
5.0 5.0 5.0
3.0
injury 2.0 2.0 1.0
15. Emergency treatment for nonoccupational illness or injury 16. Retiree health program 17. Regular inspection of plant for potential hazards 18. Alcohol control program 19. Rehabilitation program Maximal Score, Component 4:
MULTIDISCIPLINARY CONFERENCE ON HEALTH CARE ETHICS A "Conversation in the Disciplines" entitled "Ethical Dilemmas and Health Care Delivery: A Multidisciplinary Approach to Issues and Decision-Making" will be held April 25-26, 1975, at the State University of New York at Buffalo. Faculty of health sciences and related disciplines are invited to participate. For further information, write: Mila A. Aroskar, Associate Professor, School of Nursing, SUNYAB, Buffalo, NY 14214.
1975 ENVIRONMENTAL HEALTH CONFERENCE AND EXPOSITION The 1975 Environmental Health Conference and Exposition of the National Environmental Health Association will be held June 28 to July 3, 1975, at the Downtown Radisson, Minneapolis, MN. More than 1000 environmental health personnel concerned with food sanitation, waste disposal, air and water pollution, occupational safety and health, hospital and nursing home environments and other institutions, urban and rural housing, administration and planning, and education are expected to attend. Approximately 50 commercial exhibits are planned, along with educational materials and other environmental services. For more information concerning the meeting contact: the National Environmental Health Association, 1600 Pennsylvania, Denver, CO 80203.
3.0 2.0 3.0 5.0 5.0 60.0
