Regular Articles Interrater Reliability in Evaluating Trainee Interviewing Skills Bryce Templeton, M.D. Merril MacD. Allen The reliability of the evaluative judgments by psychiatric faculty of physician-trainee interviewing skills was studied. Three methods were included: global rating scales, data checklists, and a time-allotment form. Data were obtained during a training workshop for psychiatric instructors in the U.S. The authors found low interrater reliability with all three methods. The study findings were replicated at a second workshop with Canadian faculty. The authors outline some recommended modifications of obseroational systems that may help improve both the accuracy and reliability of ratings of trainee interviewing skills. The use of more accurate quantitative techniques is briefly reviewed. ~e ability of a physician to provide effec-
1
tive medical care is dependent upon the utilization of good interpersonal skills, including medical interviewing (1-4). Despite their importance, the amount of teaching and evaluation of these skills varies considerably from one U.S. medical school to another. For example, in a 1978 nationwide National Library of Medicine (NLM) survey regarding interpersonal skills instruction, Kahn et al. found that only 69% of the reporting programs employed some form of direct observation of the student with a patient (5). Few schools required formal demonstration by students of proficiency in medical interviewing skills. In the United States, the licensing authorities have been even less willing to undertake the task of evaluating these skills. Dr. Templeton is the Director of Medical Student Education in Psychiatry, Department of Psychiatry and Human Behavior, Jefferson Medical College, 1015 Walnut Street, 3rd Floor Curtis, Philadelphia, PA 19107. Merril MacD. Allen is a Quality Assurance Consultant. Address reprint requests to Bryce Templeton, M.D., at above address. Copyright © 1990 Academic Psychiatry.
'
1
tive medical care is dependent upon the utilization of good interpersonal skills, including medical interviewing (1-4). Despite their importance, the amount of teaching and evaluation of these skills varies considerably from one U.S. medical school to another. For example, in a 1978 nationwide National Library of Medicine (NLM) survey regarding interpersonal skills instruction, Kahn et al. found that only 69% of the reporting programs employed some form of direct observation of the student with a patient (5). Few schools required formal demonstration by students of proficiency in medical interviewing skills. In the United States, the licensing authorities have been even less willing to undertake the task of evaluating these skills. Dr. Templeton is the Director of Medical Student Education in Psychiatry, Department of Psychiatry and Human Behavior, Jefferson Medical College, 1015 Walnut Street, 3rd Floor Curtis, Philadelphia, PA 19107. Merril MacD. Allen is a Quality Assurance Consultant. Address reprint requests to Bryce Templeton, M.D., at above address. Copyright © 1990 Academic Psychiatry.
'
