ant. 1. Radiation
Oncobgy
Biol.
Phys..
1976, Vol.
1. pp. 323-327.
Pcrpamon
PEW.
Printed
in the U.S.A.
@Special Features CONTINUING EDUCATION AND RECERTIFICATION-A CRITICAL LINK FREDRIC D. BURG, M.D.? 3930 Chestnut Street, Philadelphia, PA 19104,U.S.A. For many reasons the medical profession is being challenged to develop a system for assuring itself and the American public that individuals certffkd in the various medical discipHneshave maintab~ed their competency. This paper presents some general cbamcteristics of such a system and suggests various methods wbkh might be used periodically to evaluate physician competency. The need for each medical dkcipfine to face this chafknge and make a commitment of time and resources is also discussed. Continuing education, Recert&ation, Peer review.
INTRODUCTION The goal of all practicing physicians should be to provide their patients with highquality, upto-date care in an efficient and well organized manner. Because there have been major advances in the diagnosis and management of medical problems over the past decade, physicians who graduated from medical school ten or more years ago have had to participate in educational experiences in order to keep abreast of the new knowledge necessary to care for their patients adequately. Until rather recently both the public and the medical profession have accepted the premise that because physicians are professionals they will all automatically maintain their competency to practice high-quality medicine over time. However, statements such as the following question this premise: “Should a doctor, whose license to practice usually lasts for life, face relicensing? And should medical specialists periodically prove they know the latest’ about brain surgery, broken bone repair or cancer cures? Yes, they certainly should, say an increasing number of consumerists and other critics of medicine.“’ and L‘ . . . . that the American Board of Medical
Specialities (ABMS) adopt in principal, and urge concurrence on its member boards the policy that voluntary, periodic recertification of medical specialists become an integral part of all national medical specialty programs and further, the ABMS establish a reasonable deadline when voluntary, periodic certification of medical specialists will have become a standard policy of all Member boards.“’ These attitudes would suggest that the medical profession is being chafienged to demonstrate to their peers and the public on a regular basis that each physician has maintained his or her competency. It is my hope that this challenge will be accepted by us as physicians rather than being mandated upon us by the federal govemment.3*4 In accepting this challenge we should attempt to create a high-quality system which includes both programs of continuing education and periodic evaluation of competency.
CHARACTERISTICS OF A SYSTEM FOR CONTINUING EDUCATION AND PERIODIC EVALUATION OF COMPETENCY 1. Each practice discipline should attempt to define those skills and abilities needed by practitioners of the discipline to deliver
TDirector, Division of Graduate and Continuing Evaluation, National Board of Medical Examiners. 323
324
Radiation Oncology 0 Biology 0 Physics
acceptable health care to patients who seek their services. 2. Programs of continuing education should be developed by appropriate societies, organizations, and medical centers to provide opportunities for individuals from a particular discipline to learn those skills and abilities as defined above. 3. Methods of evaluation should be used on a periodic basis to determine the degree of competency of each practttioner. In developing a system with the characteristics just described, the first and most crucial step is the generation of a list of disease states peculiar to the specified discipline and the creation of definitions of those skills and abilities needed to adequately diagnose and manage these diseases. The most recent specialty group to undertake this type of activity is pediatrics where the American Board of Pediatrics has developed a document, “Foundations for Evaluating the Competency of Pediatricians”.’ After deciding what the practitioner should be able to do, the creation of methods of education and evaluation is the next step. At this point I would like to emphasize the title of this paper-Continuing Education and Recertification-A Critical Link. By creating a system that will assure the coordinated development of an educational and evaluation system, those who participate have the following advantages: 1. By participating in the educational system the participants will know what will be expected of them in the evaluation component of the program. 2. If the participants level of performance on the evaluation component is below an acceptable level he can return to the educational component for assistance in reaching the necessary level of achievement. Parenthetically, the more accurately the evaluation component can identify specific areas of weakness the more useful it will be in directing each participant to the appropriate portion of the education system. Because
the focus of my expertise
is in the
Vol. 1, No. 3-t
evaluation of physician competency, I would like to spend the last portion of this paper discussing some principles of evaluation and some possible methodologies which might be considered in creating the evaluation component of a program such as that described above.
THE EVALUATION OF PHYSICIAN COMPETENCY As a profession, physicians should be at least as critical in their examination of the
evaluation techniques being used to determine their competency as they are of the various clinical laboratory values which they examine in deciding the status of their patients’ health. Therefore it is essential that the techniques employed for purposes of decision making regarding an individual’s competency or lack of competency are reliable and valid. A reliable method is one which provides reproducible results that vary little on the basis of the method itself. For example, in a clinical laboratory if two aliquots of blood from the same patient drawn at the same time are sent to be analyzed by the laboratory for a sodium determination, there should be negligible difference in the results reported. Similarly if an individual is tested on two carefully constructed, parallel examinations covering the same subject matter, his performance should vary slightly from examination to examination. Validity is a term which reflects the ability of a test to measure what it is designed to measure. Once again using the clinical laboratory-if a serum creatinine is a valid indicator of renal function, variations in its level should correspond with variations in other more exacting procedures which measure renal function, such as inulin clearances. Similarly in the evaluation of clinical competency a valid measure would be one that accurately measured or predicted how individuals will actually care for patients with those problems which would be encountered within their practice. Keeping these criteria in mind I would like to present a capsule summary of some types of evaluation procedures which might be of use in
Continuing education and recertification 0 F. D. BURG
the evaluation component of a program periodic evaluation of competency.
for
1. Multiple choice question (MCQ) examina tions This form of examination is presently the most widely used method for determining an individual’s knowledge of a particular subject matter area in medicine. In recent years MCQ examinations that depend upon the participants’ clinical judgement as well as factual knowledge have been constructed. Often these types of questions depend upon the use of pictorial or other forms of graphic material. In terms of reliability, MCQ examinations of about 100 items in length which have been carefully constructed and reviewed by panels of experts reach reliabilities which are high enough to make fair decisions about an individual’s level of performance. From the prospective of validity it is important to recognize that an MCQ examination is an assessment of what a person knows and not what a person does. It gives an indirect indication of how one might perform. However, it would seem reasonable to assume that the information needed to act correctly should be known if an individual is going to care for patients properly. One recent innovation in MCQ examinations for recertification purposes was developed by the American Board of Internal Medicine and the American College of Physicians. About eight months before the examination each participant in the program was sent a syllabus of material which would serve as the basic content for the recertification examination. Included with the syllabus was a series of short MCQ examinations in each of a number of subject matter areas. In October 1974, all those who participated in the program were given a closed book MCQ examination. The majority of questions on this examination were related directly to the material within the previously distributed syllabus.6 Another possible approach to the use of MCQ examinations for recertification is related to the problem of variable types of practices in a particular discipline. For example, in pediatrics, it is possible for a board
325
certified pediatrician to have a practice where a majority of the patients who are seen have learning disabilities, while another pediatrician might be seeing primarily healthy infants and children. The ability to construct examinations customized to a particular individual’s practice led the American Academy of Pediatrics and the American Board of Pediatrics to develop an MCQ examination format where each participant could select from a pool of approximately 500 items those items which were relevant to their practice. Out of the 500 items on the examination, 138 were judged as being relevant to practice by 70% or more of the approximately 1000 participants.’ 2. Patient management problem (P&P) examinations This is a relatively new form of paper and pencil testing which was developed along the lines of linear and branched forms of programmed instruction.89 The PMP examination allows each participant to simulate their approach to the diagnosis and management of a patient with a particular problem in a sequential manner. Through the use of an erasure technique the participant gets immediate feedback concerning each course of action selected. For example, if a chest X-ray is ordered, upon erasing a portion of the answer book, the candidate could be given an X-ray report or be directed to a particular X-ray in a pictorial atlas which would be part of the examination materials. Through various scoring strategies it is possible to determine characteristics of an individual’s clinical style in problem solving and management. One approach to scoring allows for the identification of an individual who orders too many inappropriate laboratory studies or who manages patients by making many incorrect decisions. The PMP examination in a linear format has been shown to have an acceptable level of reliability.* Other studies have indicated that this form of testing is also valid in certain ways.“’ One of the most attractive aspects of PMP examinations is the generally positive reaction of examinees to this form of evaluation. The reason for this positive reaction is related to the
326
Radiation Oncology 0 Biology 0 Physics
examination’s practice.
attempt
to simulate clinical
3. Use of medical record audits Although in the early stages of study, the use of medical record audits as a means for determining the adequacy of an individual’s approach to a variety of clinical problems may lead to a more valid approach to making decisions about an individual’s competency. Presently the National Board of Medical Examiners in conjunction with the American Board of Pediatrics is in the process of determining the reliability of such a system in evaluating the performance of pediatric residents in an outpatient setting. The focus of the study is to ascertain the amount of variability obtained by individual residents in caring for patients with the same diagnosis. Also, the effects of clinical environment and training of raters as major variables effecting reliability will be analyzed. 4. Use of peer review mechanism The National Board of Medical Examiners also is involved presently in the study of the usefulness of behaviorally keyed rating scales for use in determining the competency of physicians and physician assistants. Behaviorally keyed rating scales are scales which use defined, specific types of behavior as a means of aiding raters in describing an individual rather than a general description. A behaviorally keyed scale is one where specifics of record keeping behavior are described, with examples of good behavior and of poor behavior, as contrasted to a non-behaviorally keyed system which would ask about a general characteristic, such as compulsivity or neatness. If they are found to be reliable and valid these forms could be completed by physicians, allied health personnel and patients. Information obtained from such sources might be used to identify physicians who are weak in various areas of patient care. The consideration for the use of such an evaluation tool must await the determination of its reliability and validity. If found to be useful, there are also many important philosophical issues which must be considered. Issues such as the
Vol. 1, No. 3-4
effect on performance and relationships that would occur by knowing that one’s peers, stafI and patients might be evaluating them on a day-to-day basis. 5. Use of simulators The skills of physicians in communicating with patients might be evaluated through the use of trained actors and actresses. This too is a relatively new field and is in the process of study by various groups. The purpose would be to introduce a well-trained simulator to interact with the physician being assessed, to determine how well important information was obtained and how well instructions were communicated. This system of evaluation is dependent on the creation of valid and reliable rating forms and check lists as was previously discussed in the section on peer review mechanisms. Similarly, many of the same philosophical questions raised in the previous section would need to be considered here. CONCLUSION It is my opinion that the time has come for those of us in the medical profession to face seriously the questions of periodic evaluation of competency and intense continuing education. The opportunity for the various medical professions to have the leading voice is now available. However, if we do not begin to show signs of acting in a responsible fashion in dealing with this problem there is a very real possibility that others such as the federal government will take this responsibility upon themselves. As previously discussed there are many technical, pedagogic, and philosophical issues entwined in creating a system which will assure to the public that those delegated the responsibilities of providing health care are competent to do so. Only with careful study, thought and commitment of time and resources can these issues be sorted out. Hopefully, following such a commitment, a useful, reliable, and valid system of continuing education and periodic evaluation of competency can be created for each of the medical disciplines.
Continuing education and recertification 0 F. D. BURG
327
REFERENCES 1974 pp. l-91. 1. Lubin, J.: Do Doctors Need a Check-Up? Wall 6. Frommeyer, Street Journal, 1974, p. 8. (Wall St. Journal has W.B.: Recertification: a status no vol. nos; and on one p. only). report. Ann. Zntemal Med. 79: 441-443, 1973. 2. American Board of Medical Specialties: Pro7. Burg, F.D., Kelly, P.: Assessment by Selected ceedings, 41st Annual Meeting 29-30 March Practitioners of Questions Used in a Medical 1973, Chicago, American Board of Medical Specialty Board’s Written Examination. Abstract, p. 73, in Proceedings Thirteenth Annual specialties, 1973, pp. 35. 3. US Department of Health, Education and Conference on Research in Medical Education, 12-13 November 1974, (Sponsored by AssociaWelfare: Report on Licensure and Related Health Professional Credentialing, June 1971, tion of American Medical Colleges, Division of Washington, Department of Health, Education, Educational Measurement and Research in and Welfare, Office of Assistant Secretary for conjunction with the 85th Annual Meeting, Health and Scientific Affairs, 1971, Publication November, 1974), Chicago, Association of No HSM 72-11, pp. l-250. American Medical Colleges, 1974, pp. l-288. 4. US Congress, Senate: A Bill to amend the 8. Hubbard, J.P., Levit E.J., Schumacher, C.F., et Public Health Service Act to revise and extend al: An objective evaluation of clinical compethe programs of assistance under Title VZZfor tence. 4 June 1%5, N. Engl. J. Med. 272: training in the health and allied health projes1321-1328. sions, to revise the National Health Service 9. McGuire, C.H., Babbott, D.: Simulation techniCorps program and the National Health Service que in the measurement of problem solving Corps scholarship training program, and for skills. J. Educ. Meas. 4: I-10, Spring 1967. other purposes. S. 358593rd Congress, Second 10. Schumacher, C.F.: Validation of the American Board of Internal Medicine Written ExaminaSession, 1974, pp. l-103. tion. Ann. Intern. Med. 78: 131-135, January 5. American Board of Pediatrics, Foundations for Evaluating the Competency of Pediatricians. 1973. Chicago, The American Board of Pediatrics, QUESTION AND ANSWER SESSION points for recertification on the basis of attendance Following are some of the questions asked of the at continuing education courses? author at a meeting of the Teachers of Academic Radiation Therapy, Key Biscayne, Florida, 29 October 1974. Frederic Burg, M.D. The problem is whether or not attendance can be John Maier, M.D. equated with learning. I feel that it certainly makes The system you described sounds very complex sense to credit someone for participating in a and certainly will require a great deal of personnel learning experience and that this credit might be a effort to implement. I wonder how those of us with component part of a recertification program, if all our time already committed can possibly do all there is some evidence on the part of the participant that is needed to develop a valid and reliable system that learning has taken place. This could be done by of continuing education and periodic evaluation of use of a pretest and post-test or post-test alone. competency? Frederic Burg, M.D. Your question is extremely pertinent. 1 believe that your discipline, radiology,must decide if this activity is a high priority item at the present time. If judged to be of high priority, then I believe as a first step it would be wise to identify and fund one interested Radiologist to come and work with us on the development of this system. I believe the magnitude of the problem requires initially at least the commitment of one full-time person. Seymour Levitt, M.D. What do you think about the value of receiving
Theodore Phillips, M.D. What about the evaluation of skills which are needed to work with patients who are undergoing radiation therapy and which cannot be evaluated by written or oral examinations? Frederic Burg, M.D. This is a most complicated problem, but one which needs careful study. It is a matter for research in the development of objective rating scales for use by trained observers. I think reliable and valid tools for assessing these aspects of competency could be developed with appropriate effort.
Oncobgy
Biol.
Phys..
1976, Vol.
1. pp. 323-327.
Pcrpamon
PEW.
Printed
in the U.S.A.
@Special Features CONTINUING EDUCATION AND RECERTIFICATION-A CRITICAL LINK FREDRIC D. BURG, M.D.? 3930 Chestnut Street, Philadelphia, PA 19104,U.S.A. For many reasons the medical profession is being challenged to develop a system for assuring itself and the American public that individuals certffkd in the various medical discipHneshave maintab~ed their competency. This paper presents some general cbamcteristics of such a system and suggests various methods wbkh might be used periodically to evaluate physician competency. The need for each medical dkcipfine to face this chafknge and make a commitment of time and resources is also discussed. Continuing education, Recert&ation, Peer review.
INTRODUCTION The goal of all practicing physicians should be to provide their patients with highquality, upto-date care in an efficient and well organized manner. Because there have been major advances in the diagnosis and management of medical problems over the past decade, physicians who graduated from medical school ten or more years ago have had to participate in educational experiences in order to keep abreast of the new knowledge necessary to care for their patients adequately. Until rather recently both the public and the medical profession have accepted the premise that because physicians are professionals they will all automatically maintain their competency to practice high-quality medicine over time. However, statements such as the following question this premise: “Should a doctor, whose license to practice usually lasts for life, face relicensing? And should medical specialists periodically prove they know the latest’ about brain surgery, broken bone repair or cancer cures? Yes, they certainly should, say an increasing number of consumerists and other critics of medicine.“’ and L‘ . . . . that the American Board of Medical
Specialities (ABMS) adopt in principal, and urge concurrence on its member boards the policy that voluntary, periodic recertification of medical specialists become an integral part of all national medical specialty programs and further, the ABMS establish a reasonable deadline when voluntary, periodic certification of medical specialists will have become a standard policy of all Member boards.“’ These attitudes would suggest that the medical profession is being chafienged to demonstrate to their peers and the public on a regular basis that each physician has maintained his or her competency. It is my hope that this challenge will be accepted by us as physicians rather than being mandated upon us by the federal govemment.3*4 In accepting this challenge we should attempt to create a high-quality system which includes both programs of continuing education and periodic evaluation of competency.
CHARACTERISTICS OF A SYSTEM FOR CONTINUING EDUCATION AND PERIODIC EVALUATION OF COMPETENCY 1. Each practice discipline should attempt to define those skills and abilities needed by practitioners of the discipline to deliver
TDirector, Division of Graduate and Continuing Evaluation, National Board of Medical Examiners. 323
324
Radiation Oncology 0 Biology 0 Physics
acceptable health care to patients who seek their services. 2. Programs of continuing education should be developed by appropriate societies, organizations, and medical centers to provide opportunities for individuals from a particular discipline to learn those skills and abilities as defined above. 3. Methods of evaluation should be used on a periodic basis to determine the degree of competency of each practttioner. In developing a system with the characteristics just described, the first and most crucial step is the generation of a list of disease states peculiar to the specified discipline and the creation of definitions of those skills and abilities needed to adequately diagnose and manage these diseases. The most recent specialty group to undertake this type of activity is pediatrics where the American Board of Pediatrics has developed a document, “Foundations for Evaluating the Competency of Pediatricians”.’ After deciding what the practitioner should be able to do, the creation of methods of education and evaluation is the next step. At this point I would like to emphasize the title of this paper-Continuing Education and Recertification-A Critical Link. By creating a system that will assure the coordinated development of an educational and evaluation system, those who participate have the following advantages: 1. By participating in the educational system the participants will know what will be expected of them in the evaluation component of the program. 2. If the participants level of performance on the evaluation component is below an acceptable level he can return to the educational component for assistance in reaching the necessary level of achievement. Parenthetically, the more accurately the evaluation component can identify specific areas of weakness the more useful it will be in directing each participant to the appropriate portion of the education system. Because
the focus of my expertise
is in the
Vol. 1, No. 3-t
evaluation of physician competency, I would like to spend the last portion of this paper discussing some principles of evaluation and some possible methodologies which might be considered in creating the evaluation component of a program such as that described above.
THE EVALUATION OF PHYSICIAN COMPETENCY As a profession, physicians should be at least as critical in their examination of the
evaluation techniques being used to determine their competency as they are of the various clinical laboratory values which they examine in deciding the status of their patients’ health. Therefore it is essential that the techniques employed for purposes of decision making regarding an individual’s competency or lack of competency are reliable and valid. A reliable method is one which provides reproducible results that vary little on the basis of the method itself. For example, in a clinical laboratory if two aliquots of blood from the same patient drawn at the same time are sent to be analyzed by the laboratory for a sodium determination, there should be negligible difference in the results reported. Similarly if an individual is tested on two carefully constructed, parallel examinations covering the same subject matter, his performance should vary slightly from examination to examination. Validity is a term which reflects the ability of a test to measure what it is designed to measure. Once again using the clinical laboratory-if a serum creatinine is a valid indicator of renal function, variations in its level should correspond with variations in other more exacting procedures which measure renal function, such as inulin clearances. Similarly in the evaluation of clinical competency a valid measure would be one that accurately measured or predicted how individuals will actually care for patients with those problems which would be encountered within their practice. Keeping these criteria in mind I would like to present a capsule summary of some types of evaluation procedures which might be of use in
Continuing education and recertification 0 F. D. BURG
the evaluation component of a program periodic evaluation of competency.
for
1. Multiple choice question (MCQ) examina tions This form of examination is presently the most widely used method for determining an individual’s knowledge of a particular subject matter area in medicine. In recent years MCQ examinations that depend upon the participants’ clinical judgement as well as factual knowledge have been constructed. Often these types of questions depend upon the use of pictorial or other forms of graphic material. In terms of reliability, MCQ examinations of about 100 items in length which have been carefully constructed and reviewed by panels of experts reach reliabilities which are high enough to make fair decisions about an individual’s level of performance. From the prospective of validity it is important to recognize that an MCQ examination is an assessment of what a person knows and not what a person does. It gives an indirect indication of how one might perform. However, it would seem reasonable to assume that the information needed to act correctly should be known if an individual is going to care for patients properly. One recent innovation in MCQ examinations for recertification purposes was developed by the American Board of Internal Medicine and the American College of Physicians. About eight months before the examination each participant in the program was sent a syllabus of material which would serve as the basic content for the recertification examination. Included with the syllabus was a series of short MCQ examinations in each of a number of subject matter areas. In October 1974, all those who participated in the program were given a closed book MCQ examination. The majority of questions on this examination were related directly to the material within the previously distributed syllabus.6 Another possible approach to the use of MCQ examinations for recertification is related to the problem of variable types of practices in a particular discipline. For example, in pediatrics, it is possible for a board
325
certified pediatrician to have a practice where a majority of the patients who are seen have learning disabilities, while another pediatrician might be seeing primarily healthy infants and children. The ability to construct examinations customized to a particular individual’s practice led the American Academy of Pediatrics and the American Board of Pediatrics to develop an MCQ examination format where each participant could select from a pool of approximately 500 items those items which were relevant to their practice. Out of the 500 items on the examination, 138 were judged as being relevant to practice by 70% or more of the approximately 1000 participants.’ 2. Patient management problem (P&P) examinations This is a relatively new form of paper and pencil testing which was developed along the lines of linear and branched forms of programmed instruction.89 The PMP examination allows each participant to simulate their approach to the diagnosis and management of a patient with a particular problem in a sequential manner. Through the use of an erasure technique the participant gets immediate feedback concerning each course of action selected. For example, if a chest X-ray is ordered, upon erasing a portion of the answer book, the candidate could be given an X-ray report or be directed to a particular X-ray in a pictorial atlas which would be part of the examination materials. Through various scoring strategies it is possible to determine characteristics of an individual’s clinical style in problem solving and management. One approach to scoring allows for the identification of an individual who orders too many inappropriate laboratory studies or who manages patients by making many incorrect decisions. The PMP examination in a linear format has been shown to have an acceptable level of reliability.* Other studies have indicated that this form of testing is also valid in certain ways.“’ One of the most attractive aspects of PMP examinations is the generally positive reaction of examinees to this form of evaluation. The reason for this positive reaction is related to the
326
Radiation Oncology 0 Biology 0 Physics
examination’s practice.
attempt
to simulate clinical
3. Use of medical record audits Although in the early stages of study, the use of medical record audits as a means for determining the adequacy of an individual’s approach to a variety of clinical problems may lead to a more valid approach to making decisions about an individual’s competency. Presently the National Board of Medical Examiners in conjunction with the American Board of Pediatrics is in the process of determining the reliability of such a system in evaluating the performance of pediatric residents in an outpatient setting. The focus of the study is to ascertain the amount of variability obtained by individual residents in caring for patients with the same diagnosis. Also, the effects of clinical environment and training of raters as major variables effecting reliability will be analyzed. 4. Use of peer review mechanism The National Board of Medical Examiners also is involved presently in the study of the usefulness of behaviorally keyed rating scales for use in determining the competency of physicians and physician assistants. Behaviorally keyed rating scales are scales which use defined, specific types of behavior as a means of aiding raters in describing an individual rather than a general description. A behaviorally keyed scale is one where specifics of record keeping behavior are described, with examples of good behavior and of poor behavior, as contrasted to a non-behaviorally keyed system which would ask about a general characteristic, such as compulsivity or neatness. If they are found to be reliable and valid these forms could be completed by physicians, allied health personnel and patients. Information obtained from such sources might be used to identify physicians who are weak in various areas of patient care. The consideration for the use of such an evaluation tool must await the determination of its reliability and validity. If found to be useful, there are also many important philosophical issues which must be considered. Issues such as the
Vol. 1, No. 3-4
effect on performance and relationships that would occur by knowing that one’s peers, stafI and patients might be evaluating them on a day-to-day basis. 5. Use of simulators The skills of physicians in communicating with patients might be evaluated through the use of trained actors and actresses. This too is a relatively new field and is in the process of study by various groups. The purpose would be to introduce a well-trained simulator to interact with the physician being assessed, to determine how well important information was obtained and how well instructions were communicated. This system of evaluation is dependent on the creation of valid and reliable rating forms and check lists as was previously discussed in the section on peer review mechanisms. Similarly, many of the same philosophical questions raised in the previous section would need to be considered here. CONCLUSION It is my opinion that the time has come for those of us in the medical profession to face seriously the questions of periodic evaluation of competency and intense continuing education. The opportunity for the various medical professions to have the leading voice is now available. However, if we do not begin to show signs of acting in a responsible fashion in dealing with this problem there is a very real possibility that others such as the federal government will take this responsibility upon themselves. As previously discussed there are many technical, pedagogic, and philosophical issues entwined in creating a system which will assure to the public that those delegated the responsibilities of providing health care are competent to do so. Only with careful study, thought and commitment of time and resources can these issues be sorted out. Hopefully, following such a commitment, a useful, reliable, and valid system of continuing education and periodic evaluation of competency can be created for each of the medical disciplines.
Continuing education and recertification 0 F. D. BURG
327
REFERENCES 1974 pp. l-91. 1. Lubin, J.: Do Doctors Need a Check-Up? Wall 6. Frommeyer, Street Journal, 1974, p. 8. (Wall St. Journal has W.B.: Recertification: a status no vol. nos; and on one p. only). report. Ann. Zntemal Med. 79: 441-443, 1973. 2. American Board of Medical Specialties: Pro7. Burg, F.D., Kelly, P.: Assessment by Selected ceedings, 41st Annual Meeting 29-30 March Practitioners of Questions Used in a Medical 1973, Chicago, American Board of Medical Specialty Board’s Written Examination. Abstract, p. 73, in Proceedings Thirteenth Annual specialties, 1973, pp. 35. 3. US Department of Health, Education and Conference on Research in Medical Education, 12-13 November 1974, (Sponsored by AssociaWelfare: Report on Licensure and Related Health Professional Credentialing, June 1971, tion of American Medical Colleges, Division of Washington, Department of Health, Education, Educational Measurement and Research in and Welfare, Office of Assistant Secretary for conjunction with the 85th Annual Meeting, Health and Scientific Affairs, 1971, Publication November, 1974), Chicago, Association of No HSM 72-11, pp. l-250. American Medical Colleges, 1974, pp. l-288. 4. US Congress, Senate: A Bill to amend the 8. Hubbard, J.P., Levit E.J., Schumacher, C.F., et Public Health Service Act to revise and extend al: An objective evaluation of clinical compethe programs of assistance under Title VZZfor tence. 4 June 1%5, N. Engl. J. Med. 272: training in the health and allied health projes1321-1328. sions, to revise the National Health Service 9. McGuire, C.H., Babbott, D.: Simulation techniCorps program and the National Health Service que in the measurement of problem solving Corps scholarship training program, and for skills. J. Educ. Meas. 4: I-10, Spring 1967. other purposes. S. 358593rd Congress, Second 10. Schumacher, C.F.: Validation of the American Board of Internal Medicine Written ExaminaSession, 1974, pp. l-103. tion. Ann. Intern. Med. 78: 131-135, January 5. American Board of Pediatrics, Foundations for Evaluating the Competency of Pediatricians. 1973. Chicago, The American Board of Pediatrics, QUESTION AND ANSWER SESSION points for recertification on the basis of attendance Following are some of the questions asked of the at continuing education courses? author at a meeting of the Teachers of Academic Radiation Therapy, Key Biscayne, Florida, 29 October 1974. Frederic Burg, M.D. The problem is whether or not attendance can be John Maier, M.D. equated with learning. I feel that it certainly makes The system you described sounds very complex sense to credit someone for participating in a and certainly will require a great deal of personnel learning experience and that this credit might be a effort to implement. I wonder how those of us with component part of a recertification program, if all our time already committed can possibly do all there is some evidence on the part of the participant that is needed to develop a valid and reliable system that learning has taken place. This could be done by of continuing education and periodic evaluation of use of a pretest and post-test or post-test alone. competency? Frederic Burg, M.D. Your question is extremely pertinent. 1 believe that your discipline, radiology,must decide if this activity is a high priority item at the present time. If judged to be of high priority, then I believe as a first step it would be wise to identify and fund one interested Radiologist to come and work with us on the development of this system. I believe the magnitude of the problem requires initially at least the commitment of one full-time person. Seymour Levitt, M.D. What do you think about the value of receiving
Theodore Phillips, M.D. What about the evaluation of skills which are needed to work with patients who are undergoing radiation therapy and which cannot be evaluated by written or oral examinations? Frederic Burg, M.D. This is a most complicated problem, but one which needs careful study. It is a matter for research in the development of objective rating scales for use by trained observers. I think reliable and valid tools for assessing these aspects of competency could be developed with appropriate effort.
