346
ACCREDITATION OF CONTINUING MEDICAL EDUCATION AND PUBLIC CREDIBILITY LEONARD S. STEIN, Ph.D Executive Director Illinois Council on Continuing Medical Education Chicago, Illinois
TaH E growth of mandatory continuing medical education (CME)-for renewal of the medical license and membership in medical societies, or both-has aroused serious criticism of the CME accreditation system devised by the American Medical Association (AMA) in the 1960s. On July 1, 1977 responsibility for CME accreditation shifted to the new Liaison Committee on Continuing Medical Education; it promptly began to consider these criticisms and consider necessary changes. Credibility of CME accreditation now depends on the responses offered by the Liaison Committee to these criticisms. BACKGROUND AND GROWING DISSATISFACTION
The AMA exhibited wise foresight in the 1960s when it inaugurated a quality-assurance system for continuing medical education-another major step in its continuing effort to improve medical education, comparable to its efforts in the accreditation of medical schools, internships, and residency programs earlier in the century. The CME system consisted of two parts: First, accreditation of institutions that offer CME on the basis of the Essentials for Approved Programs in Continuing Medical Education adopted by the House of Delegates in 1967; second, the individual physician counterpart to institutional accreditation, The Physician's Recognition Award (PRA) initiated in 1969, granted upon earning 150 hours of CME credit over a three-year period. 1 The views expressed herein are the author's and they reflect neither the official position of the Illinois Council on Continuing Medical Education nor of its sponsors, the Illinois State Medical Society and the eight Illinois medical schools. While the author takes full responsibility for this paper, he offers gratitude to two physicians whose comments on an early draft were immensely helpful: Donald F. Pochyly, M.D., M.Ed., professor of health sciences education, University of Health Sciences/The Chicago Medical School; and Jacob R. Suker, M.D., associate dean for graduate education, Northwestern University Medical School.
Bull. N.Y. Acad. Med.
CONTINUING MEDICAL EDUCATION
l~
~
~
CNIUN
EDCLE
U
347 AI
N4
The AMA system gained wide acceptance in the following decade. As of June 1, 1978, 13 state and specialty societies and six state licensing authorities accepted The Physician's Recognition Award, as meeting stated CME requirements, and another 16 societies and authorities had adopted CME requirements based on the model of the PRA.2 Growing dissatisfaction, however, suggests that this wide acceptance occurred because the AMA system was the only one available. Here are major examples of that dissatisfaction: As the first national specialty society to establish CME requirements for membership, the American Academy of Family Physicians (AAFP) developed a course-approval system in the late 1940s to assure the quality of CME offered to its members.3 After the AMA adopted the Essentials, AAFP formally applied for and received accreditation, but continues to demand that CME sponsors seek AAFP approval for each course creditable toward AAFP's membership education requirement. In January 1976 the American College of Emergency Physicians went a step further and instituted "the ACEP Category 1 course approval process," calling it an "accreditation system . . . to complement the AMA approval program..." that asks CME sponsors accredited by AMA also to seek advance approval by ACEP (and pay a $35 application fee) for a CME course before it may be designated as ACEP Category 1.4 Responding to a protest against another approval procedure for CME, the ACEP president wrote in mid-1977: It appears that we are rapidly reaching a point where physicians will obtain the PRA as an adjunct to meeting the membership requirements of a specialty society .. . .We are moving toward a situation where CME accreditation, as now practiced, will not be acceptable to either the public or the medical professionals.5
Other specialty societies share the concern of the family physician and the emergency physician. In the February 1977 workshop on continuing education sponsored by the Council of Medical Specialty Societies, strong questions were raised as to whether the PRA system is relevant to specialty societies' concern to maintain their members' professional competence. Several medical societies and state licensing authorities that impose CME requirements reject the AMA system in whole or in part. These have established regulations significantly different from those of The Physician's Recognition Award: California Medical Association, North Carolina Medical Society, Oregon Medical Association, Medical Society of Virginia, American College of Obstetricians and Gynecologists, AmerVol. 55, No. 3, March 1979
348
348
L. S. STEIN STEIN
ican Society of Clinical Pathologists and College of American Pathologists; and state licensing authorities in California, Illinois, Kentucky, Michigan, Rhode Island, and Washington.2'" At AMA's fifth biennial conference on CME in October 1976 the workshop groups on The Physician's Recognition Award agreed unanimously that grant of the award implies nothing about clinical competence.7 REASONS FOR THE GROWING DISSATISFACTION
The basic flaw in the AMA system is that its two parts, institutional accreditation and The Physician's Recognition Award, became so intermingled that the purpose of accreditation- to identify those organizations that "have the ability to provide meaningful CME for certain physician groups' '8-became obscured by the categories of credit announced for earning the PRA. For example, the Commission on Education Report to the 1977 House of Delegates of the Medical Society of the State of New York describes the Commission's progress in developing methods to recommend "approval of those institutions . . . which wish to designate their programs under the AMA-PRA program as. . . Category 1 . . . ." Later, the report recommends six methods by which physicians might satisfy the society's CME membership requirement, concluding: "Now that MSSNY has activated the AMA Physician's Recognition Award, the committee has been at work, etc., etc."9 Nowhere does the report mention accreditation. This basic flaw was thrust into public prominence after July 1977, when the system was divided and the Liaison Committee assumed responsibility for accreditation but not for the PRA. The committee quickly discovered that accreditation per se did not cover a variety of crucial points, including who is eligible for accreditation, definition of CME credit and the unit of credit, categories of credit for differing kinds of activities, and responsibilities of accredited sponsors. These and related matters are covered by rules governing The Physician's Recognition Award. Thus, the liaison committee found itself with only limited authority over the quality of continuing medical education. OTHER FLAWS OF CME ACCREDITATION
Under AMA administration, implementation of the Essentials-that is, accreditation of institutional sponsors-involved an elaborate application and review procedure, including on-site inspection. This implementation Bull. N.Y. Acad. Med.
CONTINUING MEDICAL EDUCATION
-~~~~~OTNIGMEIA
DCTO
349 4
was not conducted, however, in a manner consistent with the stated vigorous requirements of the Essentials. Here are major examples: The report form prescribed for surveyors who made site visits for AMA emphasized organizational, administrative, and budgetary issues rather than educational quality. Perhaps the most important feature of this flaw was AMA's failure to demand that an accredited sponsor demonstrate that its CME offerings resulted in physician learning, although the Essentials are explicit on the importance of evaluation of learner achievement. The AMA relied on volunteer surveyors, but never developed standards by which to select members for the panel of surveyors, never offered these volunteers instructions except on-the-job training with experienced but also untrained surveyors, and never prescribed detailed operational guidelines or checklists by which surveyors might judge what they observed. The AMA also asked surveyors to make formal recommendations on accreditation-a practice used by no other accreditation agency in America. Any survey team reacts positively to the typically hospitable reception of a host-applicant. In the absence of detailed operational guidelines for judgment, surveyors almost always recommended accreditation, and the AMA tended inevitably to accept survey team recommendations. In late 1971 the AMA faced increased applications from hospitals and other relatively small CME sponsors and therefore invited state medical societies to accept responsibility to accredit intrastate CME programs. The AMA restricted its responsibility to medical schools and major national and regional CME sponsors. By the end of 1976 all but one AMA component societies had been approved to survey intrastate CME programs. On the surface, this division of labor was rational. In practice, this delegation of authority further lowered standards for accreditation. Lacking selection standards and training programs for surveyors and operational guidelines for implementation of the Essentials, the AMA could not offer such material to state societies. While AMA staff provided guidance whenever requested, formal supervision of state-society accreditation activities was minimal. Despite these deficiencies, the AMA Council on Medical Education almost always routinely approved recommendations from state societies. Indeed, until 1976 AMA did not even prescribe a standard form for state societies to use in reporting accreditation decisions, so that the AMA headquarters did not know the effective dates of accreditation conferred by state societies. Both for surveys conducted directly by the AMA and those conducted Vol. 55, No. 3, March 1979
350
350
L. S. STEIN L.
S.
by state societies, one must conclude that accreditation decisions varied widely in their adherence to stated requirements of the Essentials. Even the time periods for accreditation under AMA administration were unclear. On the effective starting date there were at least three definitions. The AMA Council on Medical Education defined that date as the day on which it took action to accredit an applicant. By contrast, the subcommittee on The Physician's Recognition Award accepted as Category 1 credit all CME activities certified by the sponsor as planned programs conducted any time after the site visit (assuming, of course, subsequent accreditation), a date typically some months prior to formal action by the Council on Medical Education.10 For sponsors surveyed by state societies, the effective starting date was whatever each state society chose to designate. On periods of accreditation, the AMA system prescribed alternatives that presuppose a degree of pr&cision unrealistic in an area so ambiguous as adult education: one, two, or four years, plus a one-year accreditation for proposed new programs for which acceptable plans could be displayed. In 1976 a new provision permitted a year's probationary extension of accredited status without examination in exceptional cases. In practice there was no set expiration date for accreditation. AMA never developed a routine cut-off of accreditation in the absence of reapplication, partly because it typically could not arrange a new site visit prior to expiration of the previous accreditation. Indeed, the AMA was unable to arrange any site visit, new or renewal, on less than a year's notice; site visits occurred as late as two years after presumed expiration of the prior accreditation. When the new Liaison Committee assumed responsibility for accreditation in 1977 it could not instantly correct all the mistakes of the past; its only reasonable option was to reaffirm the accreditation decisions previously made by the AMA."1 The result is that we are stuck with a large number of CME sponsors accredited on the basis of procedures not viewed as valid by important segments of the medical community. MAJOR FLAWS OF THE PHYSICIAN'S RECOGNITION AWARD*
The Award relies heavily on attendance as a measure of learning. PRA rules prescribe that Category 1 credit may be certified12 only for CME activities that fit the definition of a "planned program-a definition that *Facts about The Physician's Recognition Award are drawn from the PRA Information/Instruction Booklet 1977.12
Bull. N.Y. Acad. Med.
CONTINUING MEDICAL EDUCATION
351
restates the requirements for accreditation. Administration of the PRA, however, involves no field checks to ascertain whether credit designated as Category 1 in fact meets the prescribed definition. Many accredited CME sponsors routinely certify all scientific or clinical meetings for Category I credit. The PRA is granted in part for earning "nonaccredited credit" Category 2-a contradiction that raises serious questions about both accreditation and the PRA. If an institution has not demonstrated competence to plan effective learning programs, how can its activities carry any kind of CME credit? PRA requirements impose rigidities that fail to take account of differences in learning methods for individual physicians,13'14 e.g., limits on the amount of credit that can be earned from teaching, research and publication, self-study, and participation in medical-care evaluation.* Perhaps the worst feature of PRA requirements is the emphasis on formal instructional programs (Category 1). The implicit and unrealistic assumption behind this requirement is that physicians are no more competent than when they were medical students and therefore require supervision by qualified faculty in order to learn. This requirement strengthens the didactic tradition so strong in medical education despite ample evidence that this approach to adult teaching-learning is ineffective in changing physicians' performance,17 18 and encourages both individual physicians and CME sponsors to seek Category 1 without regard to the educational quality of CME activities.
SOME PROPOSED SOLUTIONS Solutions to the problems recited above are by no means easy. The new Liaison Committee has already appointed working groups to improve accreditation procedures, and the AMA Advisory Committee on CME is considering major changes in requirements for The Physician's Recognition Award. 19,20 These suggestions may be useful to both LCCME and AMA. IMPROVEMENTS IN CME ACCREDITATION The wisest course appears to be a totally new start based on the *On the importance of individual study to adults, see evaluation for learning, see Ashbaugh and McKean.'6
Vol. 55, No. 3, March 1979
Tough15;
on the value of medical care
35235
L..SS. STEIN TI
definition offered by DeLuca in late 1977: "In general, an accredited organization should have the ability to provide meaningful CME for certain physician groups."8 That task can best be accomplished if guided by two inter-related considerations: Criteria should focus only on standards and procedures by which the quality of physician continuing education can be assessed, without regard to the purposes for which accredited CME programs might be used, such as license renewal and membership in medical societies or grant of certificates such as The Physician's Recognition Award. The new accreditation system should be formulated and administered so that it is not merely judgmental, but also offers potential CME planners the opportunity to improve their competence to plan effective learning experiences for their colleagues. Among others, these elements should characterize a new accreditation system: Criteria for accreditation should be clearly formulated in language meaningful to physicians, based on proven principles of educational effectiveness. Shulman offers a concise summary of these principles in his paper prepared for the 1968 conference on the medical school curriculum sponsored by the Association of American Medical Colleges.21 While he focused on undergraduate medical education, his framework applies equally well to continuing education: "Entering characteristics"-the knowledge, aptitude, and attitudes and values possessed by learners when they enter a learning experience; "the application setting" -how learners are to use what they learn or the "objectives of instruction"; and "the instructional setting"-learning methods congruent with the learner's entering characteristics and the objectives of instruction. Implicit in Shulman's analysis is a fourth principle-evaluation or assessment of how well stated objectives are achieved, including feedback to individual learners. There is, of course, no contradiction between the rigor of educational excellence urged by Shulman and the flexibility necessary for the continuing education of a group of professionals as sophisticated and diverse as are physicians. Effective administration of valid criteria requires detailed procedures analogous to the Accreditation Manual for Hospitals of the Joint Commission on Accreditation of Hospitals (although, please, considerably shorter). A well-designed application form can detail the criteria for accreditation in operational form and thereby provide a learning experience for CME planners. Administrative procedures should impose the absolute Bull. N.Y. Acad. Med.
CONTINUING MEDICAL EDUCATION -~~~~~CNIUN MEIA
353
DCTO
minimum of paperwork on sponsors and permit completion of the entire accreditation or reaccreditation process within four to six months. Needed also are realistic time periods for which accreditation is granted. Given the inherent difficulty in judging the quality of any adult education program, this plan seems reasonable: Initial accreditation for two years would be granted to all applicants who exhibit reasonable program goals and promise of ability to help physicians achieve these goals. Subsequently, either full (four-year) or provisional (two-year) accreditation would be granted, the former if evidence of learner achievement is documented, and the latter if there appears to be a valid reason why two years were insufficient for such documentation and there remains promise that identified deficiencies can be corrected within two years. Any applicant that fails to comply with stated criteria either initially or upon reapplication, of course, should be denied this hallmark of educational quality. To avoid delays in the start of accredited status because of the time required for review of applications and survey reports, accreditation (when granted) should become effective as of the date of the site visit-the day as of which all relevant information about an applicant is assembled.* A particularly important area for improvement lies in the on-site inspection and the survey team report process. No accreditation program can be stronger than the competence of its surveyors; on the other hand, few surveyors can have the broad overview of the review/decision body. To improve the quality of survey team reports and thereby enable both LCCME's and state societies' review committees to make valid decisions, three changes are needed: Perhaps the most important is to redefine the role of the surveyor so that it is analogous to that of the physician who performs a physical examination on the prospective purchaser of life insurance: Report the medical facts and leave to the underwriter the responsibility to interpret those facts in the light of actuarial standards. The accreditation surveyor should report only the degree to which a CME sponsor complies with stated criteria, leaving the final decision to an independent review body. Second, to increase inter-rater reliability and minimize the effect of individual bias, surveyors need a standardized report form that details the specific items on which the *These ideas were suggested by two members of the Illinois State Medical Society's 1976 to 1979 Committee on CME Accreditation: Dean Bordeaux, M.D., M.A.(Educ.). Peoria, chairman. and H. Close Hesseltine, M.D., Chicago. The last point-initial accreditation effective as of date of site visit-was adopted by the Illinois State Medical Society in early 1977 for accreditation decisions in this state, since national guidance on this point was lacking at the time of adoption.
Vol. 55, No. 3, March 1979
354
354
L. S. STEIN
surveyor is to report and the LCCME review committee is to base its final decision. A third needed change is the development of a set of standards for selection of survey panel members plus brief training to engender a uniformity of perspective among all members of the panel. Finally, a new CME accreditation system must clearly define CME credit in a way that bears a reasonable relation to educational achievement rather than to mere hours of attendance.8 How this might be done is suggested by the current experiments of the Oregon Medical Society and of the American College of Obstetricians and Gynecologists, which award differential amounts of credit for the same time spent in differing kinds of learning activities.2 THE PHYSICIAN'S RECOGNITION AWARD
Clearly, a single national award attesting to a minimum standard of continuing education achievement would be convenient and efficient both for individual physicians and for societies and government agencies that view continuing education as evidence of physician competence. Unhappily, the PRA has failed to win sufficient credibility to be accepted as that single national award. Improvements in CME accreditation of the kind noted above would alone add substance to The Physician's Recognition Award. Beyond that, as DeLuca points out, the AMA would perform a major service by redefining award requirements in terms of the professional responsibilities of each individual physician who seeks it.8 Specifically, the award should be marked by three essential characteristics: Requirements must take account of the wide variation in the professional responsibilities of physicians, whether they be primary care or highly specialized specialists, or researchers and teachers, or health administrators. The award must represent learning achievement related to the individual physician's professional responsibilities rather than an arbitrary number of hours spent in undifferentiated educational activities. Requirements must be sufficiently flexible to take into account the special concerns of specialty societies, state licensing agencies, and others with a legitimate concern in defining physician competence. CONCLUSION
This critique would not be complete without offering high praise to AMA leaders and staff officers responsible for initiation of CME accrediBull. N.Y. Acad. Med.
CONTINUING MEDICAL EDUCATION
355
tation a decade and more ago. In retrospect, it appears that many of the faults noted above represent false starts typical of any new endeavor. The new Liaison Committee on CME has the opportunity to produce radical improvement in the CME accreditation process and therefore in the quality of continuing education offered to American physicians. The success of LCCME will vastly increase organized medicine's stature among physicians, government agencies, and the general public, with consequent increased respect for physicians as professionals who justly deserve high status in our society. Failure by the Liaison Committee will lead inevitably to loss of credibility for CME accreditation and seriously damage organized medicine's public image as the national champion of high-quality medical education. One result of that failure is likely to be further intrusion of government into medical affairs with little if any improvement in the quality of continuing physician education.
1.
2.
3.
4.
5.
6.
REFERENCES 7. Report of Fifth Biennial Conference on Ruhe, C. H. W.: The American Medical Continuing Medical Education for State Association's Program of Accreditation Medical Associations and Specialty in Continuing Medical Education. Societies, October 5-7, 1976. Chicago, Chicago, American Medical Association, 1971. American Medical Association, 1977. Ayers, J. (ed.): Continuing Medical 8. DeLuca, V. A., Jr.: Continuing medical Education Fact Sheet, June 1, 1978. education: Establishing requirements Chicago, AMA Department of Physiand coordinating teaching activities. J. cians Credentials and Qualifications, Med. Educ. 52:926-28, 1977. 1978. 9. Minutes, House of Delegates, Medical Society of the State of New York, 1977. Ruhe, C. H. W.: Governmental and N.Y. State J. Med., March 1977, pp. societal pressures for programs of con677-80. tinuing medical education. Bull. N.Y. 10. Personal letter to the author dated May Acad. Med. 51:707-18, 1975. Letter dated April 27, 1977 from Ronald 10, 1977 from Rutledge W. Howard, L. Krome, M.D., president, American M. D., director, Department of Continuing Medical Evaluation, American MedCollege of Emergency Physicians, to ical Association. Southern Illinois University School of Medicine, and enclosed ACEP "Pro- 11. Letter dated February 20, 1978 from Saul J. Farber, M. D., chairman, Liaison gram Director's Information Sheet" and "Program Director's Application for Committee on Continuing Medical EduACEP Category 1 CME Credit." cation, to Illinois State Medical Society. Letter dated July 26, 1977 from Ronald 12. The Physician's Recognition A ward, L. Krome, M.D., president, American InstructionlInformation Booklet, 1977. College of Emergency Physicians, to Chicago, American Medical AssociaSouthern Illinois University School of tion, 1977. Medicine. 13. Sivertson, S. E., Meyer, T. C., Hanson, Illinois Department of Registration and R., et al.: Individual physician profile: Education, Regulations under Medical Continuing education related to medical Licensing Act, Rule XI. Ill. Med. J. practice. J. Med. Educ. 48: 1006-12, 153:50-54, 1978. 1973.
Vol. 55, No. 3, March 1979
356
L. S. STEIN
ance. West. J. Med. 125:241-52, 1976. 14. Meyer, T. C.: Toward a continuum in medical education. Bull. N.Y. Acad. 19. Olsen, S.: Draft position paper on The Physician's Recognition Award. PreMed. 51:719-26, 1975. sented to the AMA Advisory Committee 15. Tough, A. M.: Learning Without a on CME, November 29, 1977. Teacher. Educational Research Series No. 3. Toronto, Ontario Inst. for Studies 20. Madigan, H. S.: Memorandum on Revision of Requirements for The Physiin Education, 1967. cian's Recognition Award. Prepared for 16. Ashbaugh, D. G. and McKean, R. S.: ad hoc subcommittee of the AMA AdviContinuing medical education, the sory Committee on CME, 1978. philosophy and use of audit. J.A. M.A. 21. Shulman, L. S.: Cognitive learning and 236:1485-88, 1976. the education process. The medical 17. Miller, G.: Continuing education for school curriculum. J. Med. Educ. what? J. Med. Educ. 42:320-26, 1967. (Suppl.) 45:90-100, 1970. 18. Sanazaro, P. J.: Medical audit, continuing medical education, and quality assur-
Bull. N.Y. Acad. Med.
ACCREDITATION OF CONTINUING MEDICAL EDUCATION AND PUBLIC CREDIBILITY LEONARD S. STEIN, Ph.D Executive Director Illinois Council on Continuing Medical Education Chicago, Illinois
TaH E growth of mandatory continuing medical education (CME)-for renewal of the medical license and membership in medical societies, or both-has aroused serious criticism of the CME accreditation system devised by the American Medical Association (AMA) in the 1960s. On July 1, 1977 responsibility for CME accreditation shifted to the new Liaison Committee on Continuing Medical Education; it promptly began to consider these criticisms and consider necessary changes. Credibility of CME accreditation now depends on the responses offered by the Liaison Committee to these criticisms. BACKGROUND AND GROWING DISSATISFACTION
The AMA exhibited wise foresight in the 1960s when it inaugurated a quality-assurance system for continuing medical education-another major step in its continuing effort to improve medical education, comparable to its efforts in the accreditation of medical schools, internships, and residency programs earlier in the century. The CME system consisted of two parts: First, accreditation of institutions that offer CME on the basis of the Essentials for Approved Programs in Continuing Medical Education adopted by the House of Delegates in 1967; second, the individual physician counterpart to institutional accreditation, The Physician's Recognition Award (PRA) initiated in 1969, granted upon earning 150 hours of CME credit over a three-year period. 1 The views expressed herein are the author's and they reflect neither the official position of the Illinois Council on Continuing Medical Education nor of its sponsors, the Illinois State Medical Society and the eight Illinois medical schools. While the author takes full responsibility for this paper, he offers gratitude to two physicians whose comments on an early draft were immensely helpful: Donald F. Pochyly, M.D., M.Ed., professor of health sciences education, University of Health Sciences/The Chicago Medical School; and Jacob R. Suker, M.D., associate dean for graduate education, Northwestern University Medical School.
Bull. N.Y. Acad. Med.
CONTINUING MEDICAL EDUCATION
l~
~
~
CNIUN
EDCLE
U
347 AI
N4
The AMA system gained wide acceptance in the following decade. As of June 1, 1978, 13 state and specialty societies and six state licensing authorities accepted The Physician's Recognition Award, as meeting stated CME requirements, and another 16 societies and authorities had adopted CME requirements based on the model of the PRA.2 Growing dissatisfaction, however, suggests that this wide acceptance occurred because the AMA system was the only one available. Here are major examples of that dissatisfaction: As the first national specialty society to establish CME requirements for membership, the American Academy of Family Physicians (AAFP) developed a course-approval system in the late 1940s to assure the quality of CME offered to its members.3 After the AMA adopted the Essentials, AAFP formally applied for and received accreditation, but continues to demand that CME sponsors seek AAFP approval for each course creditable toward AAFP's membership education requirement. In January 1976 the American College of Emergency Physicians went a step further and instituted "the ACEP Category 1 course approval process," calling it an "accreditation system . . . to complement the AMA approval program..." that asks CME sponsors accredited by AMA also to seek advance approval by ACEP (and pay a $35 application fee) for a CME course before it may be designated as ACEP Category 1.4 Responding to a protest against another approval procedure for CME, the ACEP president wrote in mid-1977: It appears that we are rapidly reaching a point where physicians will obtain the PRA as an adjunct to meeting the membership requirements of a specialty society .. . .We are moving toward a situation where CME accreditation, as now practiced, will not be acceptable to either the public or the medical professionals.5
Other specialty societies share the concern of the family physician and the emergency physician. In the February 1977 workshop on continuing education sponsored by the Council of Medical Specialty Societies, strong questions were raised as to whether the PRA system is relevant to specialty societies' concern to maintain their members' professional competence. Several medical societies and state licensing authorities that impose CME requirements reject the AMA system in whole or in part. These have established regulations significantly different from those of The Physician's Recognition Award: California Medical Association, North Carolina Medical Society, Oregon Medical Association, Medical Society of Virginia, American College of Obstetricians and Gynecologists, AmerVol. 55, No. 3, March 1979
348
348
L. S. STEIN STEIN
ican Society of Clinical Pathologists and College of American Pathologists; and state licensing authorities in California, Illinois, Kentucky, Michigan, Rhode Island, and Washington.2'" At AMA's fifth biennial conference on CME in October 1976 the workshop groups on The Physician's Recognition Award agreed unanimously that grant of the award implies nothing about clinical competence.7 REASONS FOR THE GROWING DISSATISFACTION
The basic flaw in the AMA system is that its two parts, institutional accreditation and The Physician's Recognition Award, became so intermingled that the purpose of accreditation- to identify those organizations that "have the ability to provide meaningful CME for certain physician groups' '8-became obscured by the categories of credit announced for earning the PRA. For example, the Commission on Education Report to the 1977 House of Delegates of the Medical Society of the State of New York describes the Commission's progress in developing methods to recommend "approval of those institutions . . . which wish to designate their programs under the AMA-PRA program as. . . Category 1 . . . ." Later, the report recommends six methods by which physicians might satisfy the society's CME membership requirement, concluding: "Now that MSSNY has activated the AMA Physician's Recognition Award, the committee has been at work, etc., etc."9 Nowhere does the report mention accreditation. This basic flaw was thrust into public prominence after July 1977, when the system was divided and the Liaison Committee assumed responsibility for accreditation but not for the PRA. The committee quickly discovered that accreditation per se did not cover a variety of crucial points, including who is eligible for accreditation, definition of CME credit and the unit of credit, categories of credit for differing kinds of activities, and responsibilities of accredited sponsors. These and related matters are covered by rules governing The Physician's Recognition Award. Thus, the liaison committee found itself with only limited authority over the quality of continuing medical education. OTHER FLAWS OF CME ACCREDITATION
Under AMA administration, implementation of the Essentials-that is, accreditation of institutional sponsors-involved an elaborate application and review procedure, including on-site inspection. This implementation Bull. N.Y. Acad. Med.
CONTINUING MEDICAL EDUCATION
-~~~~~OTNIGMEIA
DCTO
349 4
was not conducted, however, in a manner consistent with the stated vigorous requirements of the Essentials. Here are major examples: The report form prescribed for surveyors who made site visits for AMA emphasized organizational, administrative, and budgetary issues rather than educational quality. Perhaps the most important feature of this flaw was AMA's failure to demand that an accredited sponsor demonstrate that its CME offerings resulted in physician learning, although the Essentials are explicit on the importance of evaluation of learner achievement. The AMA relied on volunteer surveyors, but never developed standards by which to select members for the panel of surveyors, never offered these volunteers instructions except on-the-job training with experienced but also untrained surveyors, and never prescribed detailed operational guidelines or checklists by which surveyors might judge what they observed. The AMA also asked surveyors to make formal recommendations on accreditation-a practice used by no other accreditation agency in America. Any survey team reacts positively to the typically hospitable reception of a host-applicant. In the absence of detailed operational guidelines for judgment, surveyors almost always recommended accreditation, and the AMA tended inevitably to accept survey team recommendations. In late 1971 the AMA faced increased applications from hospitals and other relatively small CME sponsors and therefore invited state medical societies to accept responsibility to accredit intrastate CME programs. The AMA restricted its responsibility to medical schools and major national and regional CME sponsors. By the end of 1976 all but one AMA component societies had been approved to survey intrastate CME programs. On the surface, this division of labor was rational. In practice, this delegation of authority further lowered standards for accreditation. Lacking selection standards and training programs for surveyors and operational guidelines for implementation of the Essentials, the AMA could not offer such material to state societies. While AMA staff provided guidance whenever requested, formal supervision of state-society accreditation activities was minimal. Despite these deficiencies, the AMA Council on Medical Education almost always routinely approved recommendations from state societies. Indeed, until 1976 AMA did not even prescribe a standard form for state societies to use in reporting accreditation decisions, so that the AMA headquarters did not know the effective dates of accreditation conferred by state societies. Both for surveys conducted directly by the AMA and those conducted Vol. 55, No. 3, March 1979
350
350
L. S. STEIN L.
S.
by state societies, one must conclude that accreditation decisions varied widely in their adherence to stated requirements of the Essentials. Even the time periods for accreditation under AMA administration were unclear. On the effective starting date there were at least three definitions. The AMA Council on Medical Education defined that date as the day on which it took action to accredit an applicant. By contrast, the subcommittee on The Physician's Recognition Award accepted as Category 1 credit all CME activities certified by the sponsor as planned programs conducted any time after the site visit (assuming, of course, subsequent accreditation), a date typically some months prior to formal action by the Council on Medical Education.10 For sponsors surveyed by state societies, the effective starting date was whatever each state society chose to designate. On periods of accreditation, the AMA system prescribed alternatives that presuppose a degree of pr&cision unrealistic in an area so ambiguous as adult education: one, two, or four years, plus a one-year accreditation for proposed new programs for which acceptable plans could be displayed. In 1976 a new provision permitted a year's probationary extension of accredited status without examination in exceptional cases. In practice there was no set expiration date for accreditation. AMA never developed a routine cut-off of accreditation in the absence of reapplication, partly because it typically could not arrange a new site visit prior to expiration of the previous accreditation. Indeed, the AMA was unable to arrange any site visit, new or renewal, on less than a year's notice; site visits occurred as late as two years after presumed expiration of the prior accreditation. When the new Liaison Committee assumed responsibility for accreditation in 1977 it could not instantly correct all the mistakes of the past; its only reasonable option was to reaffirm the accreditation decisions previously made by the AMA."1 The result is that we are stuck with a large number of CME sponsors accredited on the basis of procedures not viewed as valid by important segments of the medical community. MAJOR FLAWS OF THE PHYSICIAN'S RECOGNITION AWARD*
The Award relies heavily on attendance as a measure of learning. PRA rules prescribe that Category 1 credit may be certified12 only for CME activities that fit the definition of a "planned program-a definition that *Facts about The Physician's Recognition Award are drawn from the PRA Information/Instruction Booklet 1977.12
Bull. N.Y. Acad. Med.
CONTINUING MEDICAL EDUCATION
351
restates the requirements for accreditation. Administration of the PRA, however, involves no field checks to ascertain whether credit designated as Category 1 in fact meets the prescribed definition. Many accredited CME sponsors routinely certify all scientific or clinical meetings for Category I credit. The PRA is granted in part for earning "nonaccredited credit" Category 2-a contradiction that raises serious questions about both accreditation and the PRA. If an institution has not demonstrated competence to plan effective learning programs, how can its activities carry any kind of CME credit? PRA requirements impose rigidities that fail to take account of differences in learning methods for individual physicians,13'14 e.g., limits on the amount of credit that can be earned from teaching, research and publication, self-study, and participation in medical-care evaluation.* Perhaps the worst feature of PRA requirements is the emphasis on formal instructional programs (Category 1). The implicit and unrealistic assumption behind this requirement is that physicians are no more competent than when they were medical students and therefore require supervision by qualified faculty in order to learn. This requirement strengthens the didactic tradition so strong in medical education despite ample evidence that this approach to adult teaching-learning is ineffective in changing physicians' performance,17 18 and encourages both individual physicians and CME sponsors to seek Category 1 without regard to the educational quality of CME activities.
SOME PROPOSED SOLUTIONS Solutions to the problems recited above are by no means easy. The new Liaison Committee has already appointed working groups to improve accreditation procedures, and the AMA Advisory Committee on CME is considering major changes in requirements for The Physician's Recognition Award. 19,20 These suggestions may be useful to both LCCME and AMA. IMPROVEMENTS IN CME ACCREDITATION The wisest course appears to be a totally new start based on the *On the importance of individual study to adults, see evaluation for learning, see Ashbaugh and McKean.'6
Vol. 55, No. 3, March 1979
Tough15;
on the value of medical care
35235
L..SS. STEIN TI
definition offered by DeLuca in late 1977: "In general, an accredited organization should have the ability to provide meaningful CME for certain physician groups."8 That task can best be accomplished if guided by two inter-related considerations: Criteria should focus only on standards and procedures by which the quality of physician continuing education can be assessed, without regard to the purposes for which accredited CME programs might be used, such as license renewal and membership in medical societies or grant of certificates such as The Physician's Recognition Award. The new accreditation system should be formulated and administered so that it is not merely judgmental, but also offers potential CME planners the opportunity to improve their competence to plan effective learning experiences for their colleagues. Among others, these elements should characterize a new accreditation system: Criteria for accreditation should be clearly formulated in language meaningful to physicians, based on proven principles of educational effectiveness. Shulman offers a concise summary of these principles in his paper prepared for the 1968 conference on the medical school curriculum sponsored by the Association of American Medical Colleges.21 While he focused on undergraduate medical education, his framework applies equally well to continuing education: "Entering characteristics"-the knowledge, aptitude, and attitudes and values possessed by learners when they enter a learning experience; "the application setting" -how learners are to use what they learn or the "objectives of instruction"; and "the instructional setting"-learning methods congruent with the learner's entering characteristics and the objectives of instruction. Implicit in Shulman's analysis is a fourth principle-evaluation or assessment of how well stated objectives are achieved, including feedback to individual learners. There is, of course, no contradiction between the rigor of educational excellence urged by Shulman and the flexibility necessary for the continuing education of a group of professionals as sophisticated and diverse as are physicians. Effective administration of valid criteria requires detailed procedures analogous to the Accreditation Manual for Hospitals of the Joint Commission on Accreditation of Hospitals (although, please, considerably shorter). A well-designed application form can detail the criteria for accreditation in operational form and thereby provide a learning experience for CME planners. Administrative procedures should impose the absolute Bull. N.Y. Acad. Med.
CONTINUING MEDICAL EDUCATION -~~~~~CNIUN MEIA
353
DCTO
minimum of paperwork on sponsors and permit completion of the entire accreditation or reaccreditation process within four to six months. Needed also are realistic time periods for which accreditation is granted. Given the inherent difficulty in judging the quality of any adult education program, this plan seems reasonable: Initial accreditation for two years would be granted to all applicants who exhibit reasonable program goals and promise of ability to help physicians achieve these goals. Subsequently, either full (four-year) or provisional (two-year) accreditation would be granted, the former if evidence of learner achievement is documented, and the latter if there appears to be a valid reason why two years were insufficient for such documentation and there remains promise that identified deficiencies can be corrected within two years. Any applicant that fails to comply with stated criteria either initially or upon reapplication, of course, should be denied this hallmark of educational quality. To avoid delays in the start of accredited status because of the time required for review of applications and survey reports, accreditation (when granted) should become effective as of the date of the site visit-the day as of which all relevant information about an applicant is assembled.* A particularly important area for improvement lies in the on-site inspection and the survey team report process. No accreditation program can be stronger than the competence of its surveyors; on the other hand, few surveyors can have the broad overview of the review/decision body. To improve the quality of survey team reports and thereby enable both LCCME's and state societies' review committees to make valid decisions, three changes are needed: Perhaps the most important is to redefine the role of the surveyor so that it is analogous to that of the physician who performs a physical examination on the prospective purchaser of life insurance: Report the medical facts and leave to the underwriter the responsibility to interpret those facts in the light of actuarial standards. The accreditation surveyor should report only the degree to which a CME sponsor complies with stated criteria, leaving the final decision to an independent review body. Second, to increase inter-rater reliability and minimize the effect of individual bias, surveyors need a standardized report form that details the specific items on which the *These ideas were suggested by two members of the Illinois State Medical Society's 1976 to 1979 Committee on CME Accreditation: Dean Bordeaux, M.D., M.A.(Educ.). Peoria, chairman. and H. Close Hesseltine, M.D., Chicago. The last point-initial accreditation effective as of date of site visit-was adopted by the Illinois State Medical Society in early 1977 for accreditation decisions in this state, since national guidance on this point was lacking at the time of adoption.
Vol. 55, No. 3, March 1979
354
354
L. S. STEIN
surveyor is to report and the LCCME review committee is to base its final decision. A third needed change is the development of a set of standards for selection of survey panel members plus brief training to engender a uniformity of perspective among all members of the panel. Finally, a new CME accreditation system must clearly define CME credit in a way that bears a reasonable relation to educational achievement rather than to mere hours of attendance.8 How this might be done is suggested by the current experiments of the Oregon Medical Society and of the American College of Obstetricians and Gynecologists, which award differential amounts of credit for the same time spent in differing kinds of learning activities.2 THE PHYSICIAN'S RECOGNITION AWARD
Clearly, a single national award attesting to a minimum standard of continuing education achievement would be convenient and efficient both for individual physicians and for societies and government agencies that view continuing education as evidence of physician competence. Unhappily, the PRA has failed to win sufficient credibility to be accepted as that single national award. Improvements in CME accreditation of the kind noted above would alone add substance to The Physician's Recognition Award. Beyond that, as DeLuca points out, the AMA would perform a major service by redefining award requirements in terms of the professional responsibilities of each individual physician who seeks it.8 Specifically, the award should be marked by three essential characteristics: Requirements must take account of the wide variation in the professional responsibilities of physicians, whether they be primary care or highly specialized specialists, or researchers and teachers, or health administrators. The award must represent learning achievement related to the individual physician's professional responsibilities rather than an arbitrary number of hours spent in undifferentiated educational activities. Requirements must be sufficiently flexible to take into account the special concerns of specialty societies, state licensing agencies, and others with a legitimate concern in defining physician competence. CONCLUSION
This critique would not be complete without offering high praise to AMA leaders and staff officers responsible for initiation of CME accrediBull. N.Y. Acad. Med.
CONTINUING MEDICAL EDUCATION
355
tation a decade and more ago. In retrospect, it appears that many of the faults noted above represent false starts typical of any new endeavor. The new Liaison Committee on CME has the opportunity to produce radical improvement in the CME accreditation process and therefore in the quality of continuing education offered to American physicians. The success of LCCME will vastly increase organized medicine's stature among physicians, government agencies, and the general public, with consequent increased respect for physicians as professionals who justly deserve high status in our society. Failure by the Liaison Committee will lead inevitably to loss of credibility for CME accreditation and seriously damage organized medicine's public image as the national champion of high-quality medical education. One result of that failure is likely to be further intrusion of government into medical affairs with little if any improvement in the quality of continuing physician education.
1.
2.
3.
4.
5.
6.
REFERENCES 7. Report of Fifth Biennial Conference on Ruhe, C. H. W.: The American Medical Continuing Medical Education for State Association's Program of Accreditation Medical Associations and Specialty in Continuing Medical Education. Societies, October 5-7, 1976. Chicago, Chicago, American Medical Association, 1971. American Medical Association, 1977. Ayers, J. (ed.): Continuing Medical 8. DeLuca, V. A., Jr.: Continuing medical Education Fact Sheet, June 1, 1978. education: Establishing requirements Chicago, AMA Department of Physiand coordinating teaching activities. J. cians Credentials and Qualifications, Med. Educ. 52:926-28, 1977. 1978. 9. Minutes, House of Delegates, Medical Society of the State of New York, 1977. Ruhe, C. H. W.: Governmental and N.Y. State J. Med., March 1977, pp. societal pressures for programs of con677-80. tinuing medical education. Bull. N.Y. 10. Personal letter to the author dated May Acad. Med. 51:707-18, 1975. Letter dated April 27, 1977 from Ronald 10, 1977 from Rutledge W. Howard, L. Krome, M.D., president, American M. D., director, Department of Continuing Medical Evaluation, American MedCollege of Emergency Physicians, to ical Association. Southern Illinois University School of Medicine, and enclosed ACEP "Pro- 11. Letter dated February 20, 1978 from Saul J. Farber, M. D., chairman, Liaison gram Director's Information Sheet" and "Program Director's Application for Committee on Continuing Medical EduACEP Category 1 CME Credit." cation, to Illinois State Medical Society. Letter dated July 26, 1977 from Ronald 12. The Physician's Recognition A ward, L. Krome, M.D., president, American InstructionlInformation Booklet, 1977. College of Emergency Physicians, to Chicago, American Medical AssociaSouthern Illinois University School of tion, 1977. Medicine. 13. Sivertson, S. E., Meyer, T. C., Hanson, Illinois Department of Registration and R., et al.: Individual physician profile: Education, Regulations under Medical Continuing education related to medical Licensing Act, Rule XI. Ill. Med. J. practice. J. Med. Educ. 48: 1006-12, 153:50-54, 1978. 1973.
Vol. 55, No. 3, March 1979
356
L. S. STEIN
ance. West. J. Med. 125:241-52, 1976. 14. Meyer, T. C.: Toward a continuum in medical education. Bull. N.Y. Acad. 19. Olsen, S.: Draft position paper on The Physician's Recognition Award. PreMed. 51:719-26, 1975. sented to the AMA Advisory Committee 15. Tough, A. M.: Learning Without a on CME, November 29, 1977. Teacher. Educational Research Series No. 3. Toronto, Ontario Inst. for Studies 20. Madigan, H. S.: Memorandum on Revision of Requirements for The Physiin Education, 1967. cian's Recognition Award. Prepared for 16. Ashbaugh, D. G. and McKean, R. S.: ad hoc subcommittee of the AMA AdviContinuing medical education, the sory Committee on CME, 1978. philosophy and use of audit. J.A. M.A. 21. Shulman, L. S.: Cognitive learning and 236:1485-88, 1976. the education process. The medical 17. Miller, G.: Continuing education for school curriculum. J. Med. Educ. what? J. Med. Educ. 42:320-26, 1967. (Suppl.) 45:90-100, 1970. 18. Sanazaro, P. J.: Medical audit, continuing medical education, and quality assur-
Bull. N.Y. Acad. Med.
