Medical Teacher
ISSN: 0142-159X (Print) 1466-187X (Online) Journal homepage: http://www.tandfonline.com/loi/imte20
Against Multiple Choice Questions Sir George Pickering To cite this article: Sir George Pickering (1979) Against Multiple Choice Questions, Medical Teacher, 1:2, 84-86 To link to this article: http://dx.doi.org/10.3109/01421597909019397
Published online: 03 Jul 2009.
Submit your article to this journal
Article views: 59
View related articles
Citing articles: 2 View citing articles
Full Terms & Conditions of access and use can be found at http://www.tandfonline.com/action/journalInformation?journalCode=imte20 Download by: [Australian National University]
Date: 05 November 2015, At: 21:38
CONTROVERSY
Against Multiple Choice Questions SIR GEORGE PICKERING Sir George Pickering, M D , FRS, LL.D, FRCP, is Emeritus Regius Professor of Medicine at Oxford University, Oxford, UK.
Medical Teacher 1979.1:84-86.
I n this article, Sir George Pickering argues the case against multiple choice questions i n reply to the article which appeared in the last issue of Medical Teacher (1979, 1 (I), 37-42).
Since my case against Multiple Choice Questions (MCQs) is educational and cultural, I must first sketch the background. I recently conducted a survey of medical education for the Nuffield Provincial Hospitals Trust (Pickering 1978). During that survey I was deeply disturbed by the fact that so many students told us they were bored and frustrated; that they were obliged to sit in lecture theatres to take notes and to learn from their notes so that they could reproduce them for their very frequent examinations; that they had neither time, opportunity nor encouragement to pursue an interest: that they were not encouraged to find out things for themselves; that they had not acquired the habit of using a library; and that they received neither encouragement nor help in expressing themselves lucidly and concisely in the English language. In short, they had been encouraged to receive and repeat dogma, and had not been taught how to learn. This is sad for two particular reasons. Medical students are better now than they ever have been. They offer a rich promise for the future if only their minds could be encouraged to grow and not to atrophy as is now the case. Postgraduate education is splendidly developed and offers excellent opportunities for acquiring specialized knowledge and expertise, provided the students know how to learn, which, alas, their undergraduate education tends not to do for them. The causes of this sorry state of affairs are many. Too many subjects, too many teachers, and particularly the obsession of teachers with committees and new-fangled notions about teaching and a neglect of the great principles of education are amongst the most important. These principles have been enunciated by some of the great men of the past, of whom I quote three. Plutarch said “The mind does not need filling up like a vessel but merely kindling like a fire”; Karl Pearson said “The true 84
aim of the teacher should be to impart an appreciation of method rather than a knowledge of facts”; and Winston Churchill maintained, “The most important thing about education is appetite. Education does not begin with the University and it certainly ought not to end there”. In effecting this sorry situation, two instruments have been pre-eminently successful, namely continuous assessment and MCQs.
Continuous Assessment Undoubtedly the best test of a person’s ability to perform a task is his ability to do it well. This is the philosophy behind continuous assessment, in which the student’s performance in his classes, seminars or practicals, or in presenting cases to his chief, are awarded marks. This can be a splendid method not only of judging a student’s fitness, but also of monitoring his progress and of correcting his deviations from what his teachers regard as his optimum performance. I came to understand this method as Master of an Oxford College, where, at the end of every term, each tutor presented his pupils to me, together with their supervisors’ report. At their best these were superb appreciations of character, intellect and habits of work. The worst was ‘satisfactory’ or even ‘satis’; a report that revealed more of the teacher than of his pupils. As a teacher of medicine I found it easy to calibrate the students’ performance during their weekly presentation of their ward patients or outpatients to me. I also enjoyed castigating them for bad habits or congratulating them for good ones. I had hoped that with such reports we might largely dispense with examinations in Oxford, except for the award honours and prizes. But I could not convince my colleagues because they distrusted most teachers’ reports. They were, of course, quite right. Too many teachers never listened to their students or examined their work. Had real continuous assessment been introduced, they would have had to. That was the real reason why I wished to see it introduced. So in Oxford and in other medical schools a great opportunity to improve relations between teachers and Medical Teacher Vol 1 No 2 1979
Medical Teacher 1979.1:84-86.
taught has been missed. What has been substituted for continuous assessment is frequent discontinuous examination. In one school I visited there were no fewer than 60 examinations in five years. These examinations are usually MCQs set by the teachers themselves. And they can be very rigorous and concerned with minutiae. In one school a paper on biochemistry had 150 MCQs. No wonder the students complained that they were being turned into data banks! Dissatisfaction with the so-called essay type question, whose marking was so capricious as to be virtually unrelated to the capacity and performance of the candidate, led to the present popularity of MCQs. The socalled essay question is merely a clumsy attempt to test so-called factual knowledge. In reality i t serves only to test ability to recall lecture notes or the pages of a favourite textbook. An example would be “Describe the symptoms, signs, complications, prognosis and treatment of ulcerative colitis”. Examiners, called upon to read one hundred or so of these dreary, monotonous scripts, could be forgiven for rewarding brevity, as when verbs were omitted from sentences and lists were given. They encouraged memorization and bad English. That these socalled essay-type questions were’ killed by the proponents of MCQs is one of the points of agreement between us. It should be noted here that questions such as the one I have just cited are not real essays. An essay, as it is usually understood, is a development of a point of view, marshalling the arguments for and against, and setting them out lucidly and in an orderly manner.
T h e MCQ The MCQcan be asked in various ways which are fully set out elsewhere (Fleming et al. 1976; Anderson 1976). A n derson’sfirst example is: A 46-year-old male patient who is moderately obese is suspected of having Cushing’s disease. The single observation which would give most support to this diagnosis would be: A. Urinary 17-oxosteroid excretion of 80 pmo1/24 hr (23 mg/24 hr). B. A midnight plasma cortisol of 555 nmol/l (20 ,ug/lOO ml) . C. A blood pressure of 170/110. D. A diabetic glucose tolerance curve. E. The presence of pink abdominal striae. (Correct answer: B)
I could quote many others, but this will serve to show the basis of my criticisms. My first criticism is that this question is artificial. When the question arises as to what is the best single test for Cushing’s disease, a frame of five alternatives does not immediately arise in one’s mind. One’s mind is blank. The natural way to ask such a question would be “What single measurement would give most support to the diagnosis of Cushing’s disease?” A second difficulty about this question is that the preciseminded student might ask, what about a plasma cortisol of 550 nmol or 560 nmol? In other words, it would have been better to say a midnight plasma cortisol of over 550 Medical Teacher Vol 1 N o 2 1979
nmol. The best students often find these questions almost unanswerable. My third objection is that it is not strictly knowledge that is being tested, but opinion or dogma. These so-called objective tests are only objective in the sense that the examiners have met and agreed on which answers are correct and which not. To this extent personal bias has been eliminated. The respect that is engendered in the student for dogma terrifies me. Are we witnessing a return to authoritarianism and the tyranny of opinion such as suppressed intellectual freedom in the late Middle Ages? Is history going to repeat itself and are we on the threshold of another Dark Age? The kind of knowledge that they test is that which a Dean of Harvard had in mind when he said “I tell my students that one of the difficulties they will face in medicine is that in 10 years’ time they will find that half of what they were taught is wrong and neither I nor any of my teachers know which half ’. A further objection to MCQs is that they tend to test trivia. The doctor faced with this patient would take all the evidence into consideration and not just one item. If he were a wise doctor he would refer the patient suspected of a rare disease like Cushing’s syndrome to an expert who would take the decision. The greatest menace to good medical practice is not ignorance but a failure to know one’s own ignorance. If one is judging competence it is important that the examination should test that and not something else. This was brought home to me nearly 50 years ago by my friend L. R. Wager. He had applied to join the next Everest Expedition (1932) and was up for his medical. He failed. He could not hold his breath long enough nor could he blow up a column of mercury high enough, tests designed in the First World War to select pilots for highaltitude flying. But, though deemed unsuitable for the assault on the mountain, the authorities took him because he was a very experienced mountaineer. When the party arrived on the mountain, several of those who had done brilliantly in the tests developed Cheyne-Stokes respiration, or palpitations and breathlessness, and had to be sent back. Wager, however, plodded on and became one of the select few who had climbed at 28,000 feet without oxygen and had survived. The tests, though quantitative and replicable, tested something different from the real issue. Might the same be true of MCQs? My greatest objection to MCQs is that in my opinion they are a hazard to the future of medical culture. It is important that we should realise that we have inherited a corpus of knowledge and an attitude to patients and their sufferings through the exertions of our predecessors. We owe the next generation an obligation that we continue that work. Our present culture in medicine has been based on curiosity, the scientific method, and precision in thought and expression. The present generation of students is not encouraged to develop any of these habits of mind. History has shown that unless such habits are preserved, they tend to vanish. I hope that those who develop the case for MCQs will discover how they can encourage curiosity and initiative, the respect for evidence and the capacity to think clearly and to write and speak clearly, for on these our future depends. 85
Reform of Examinations If I had the responsibility of reforming examinations in medicine, what would I do? In the first place I would abolish most of them and replace them with true continuous assessment based on the student’s performance in class. The examinations that I retained would each consist of two papers. One paper would consist of openended, short factual questions such as: 1. Define pulsus alternans. How would you recognize it? What is its clinical significance? 2. Outline the treatment you would prescribe for a man of 56 with severe cardiac failure, cardiac enlargement and auricular fibrillation with a ventricular rate of 120 per minute?
Medical Teacher 1979.1:84-86.
This paper would have as its preamble “All the questions should be answered. These questions are designed to test the candidate’s factual knowledge, and a tabular form of answer is appropriate”. These two questions are not artificial; they occur to a practitioner repeatedly and require an answer if the patient is to be well served. The student either knows the answer or not. To get the correct answer by chance is unlikely. The other paper would consist of a choice of an essay on one of half a dozen questions. An example is: Write an essay on one of the following subjects. Can-
didates should devote about 1+ hours of their time to this. Due note will be taken of the form of presentation and style, as well as of the matter. 1. Surgery and ethics. 2. The problems of geriatric surgery. 3. The importance of the development of accident departments. 4. The importance of the collaboration of surgeon and physician in the interests of the patient. 5 . Conditions mimicking carcinoma of the breast. To mark these answers would, of course, require more time and more trouble on the part of the examiner than does the mechanical treatment of multiple choice, but they do have the merit of testing what is important to the doctor and of encouraging scholarship and clarity of thought and expression. I hope that sometimes they may be tried on a large scale and that the methods and their results will be analysed and criticized as freely as MCQs have been. References Anderson. J . . T h e Multiple Choice Question in Medicine, Pitman Medical, London, 1976. Fleming, P. R.,Sanderson. P. H.. Stokes, J. F. and Walton, H . J.. Examhations in Medicine, Churchill Livingstone. Edinburgh, London, 1976.
Pickering. G.. Quest for Excellence in Medical Education, Oxford University Press for the Nufficld Provincial Hospitals. London and Oxford. 1978.
.
I COURSESAND I
Miss M Cyle Association for the Study of Medical Education 150b Perth Road Dundee DD14HN
Association for the Study of Medical Education, Royal College of General Practitioners
We would welcome contributions relating to forthcoming courses and conferences which are of interest to medical teachers and other educators in the health sciences. Copy should be addressed to the Managing Editor and should be submitted no later than nine weeks preceding publication date.
Association for the Study of Medical Education, T h e Royal Society of Medicine, Medical Education Section, and the Institute of Mathematics and its Applications Teaching and Learning Statistics in Medicine. University of Manchester. 20 April. For further information write to: write to:
86
Royal College of General Practitioners Meeting on Continuing Medical Education to be held on 16 May 1979. For further information write to: Miss M. Cyle Association for the Study of Medical Education 150b Perth Road Dundee DD14HN
Association for the Study of Medical Education Annual Scientific Meeting to be held at Southampton Medical School on 27 and 28 September 1979. For further information write to: Miss M. Cyle Association for the Study of Medical Education 150b Perth Road Dundee DD14HN
Joint Committee on Contraception Conference for doctors interested in teaching family planning to be held at the Royal College of Obstetricians and
Gynaecologists on 1 June 1979. For further information write to: Dr Barbara Law Chairman Joint Committee on Contraception 27 Sussex Place Regents Park London N W 1 4RG
Centre for Medical Education A course for teachers in the UK, undergraduate and postgraduate. New Approaches to Teaching and Learning, University of Dundee on 3 to 7 September 1979. For further information write to: Professor R. M. Harden Centre for Medical Education Level 8 Ninewells Hospital and Medical School Dundee DDl 9SY
Association for Medical Education in Europe Annual Meeting to be held in Athens on 18 to 20 September 1979. For further information write to: Mrs Ann Combe AMEE Department of Psychiatry Royal Edinburgh Hospital Morningside Park Edinburgh
Medical Teacher Vol 1 No 2 1979