142
Design of the ICD-IO Field Trial in German-Speaking Countries H. Dilling. H. J. Freyberger. C. P. Malchow Departrnent ofPsychiatry, (Director: Prof. Dr. H. Dilling), Lübeck Medical University, Lübeck, FRG
The study design of the international multicentric field trial on ICD-I0, chapter V (F) "Mental, Behavioural and Developmental Disorders: Clinical Descriptions and Diagnostic Guidelines" (1987 draft) including some modifications for the study in German-speaking countries, is described. The various stages of the field trial, instruments used, and the data base, are presented.
Introduction A preliminary version of ICD-l 0 chapter V (F) ("Mental, Behavioural and Developmental Disorders") has existed since 1984 and has undergone several modifications. In 1987/88 a field study was proposed by the Mental Health Division ofthe World Health Organization in Geneva with the aim of gaining experience with the current version (1987 draft; WHO, 1987) of the "Clinical Descriptions and Diagnostic Guidelines" of this chapter. The three major objectives of the field trial were (i) Assessment of the ease of understanding and ease of use of the new classification ("feasibility"). (ii) Assessment of the extent to which the new system adequately describes the disorders of various types of psychiatrie patients in routine clinical practice ("suitability, goodness offit"). And (iii) Assessment of the extent and type of agreement achieved between independent diagnosticians in different centres ("inter-rater reliability"). These aims should be accomplished by a large international study in 54 countries with different psychiatrie schools, so as to compare the classification under different social conditions and in different languages. The WHO was helped by the experience of the American Psychiatrie Association dating back to 1977, when during a two-year period DSM-III (APA, 1980) was tested by a
Pharmacopsychiatry 23 (1990) 142-145 (Supplement) C Georg Thieme Verlag Stuttgart·New York
Untersuchungsplan der ICD-I6-Feldstudie In dendeutschspncbigen Lindern Der Untersuchungsplan der internationalen multizentrischen Felduntersuchung zur Einführung des Kapitels V (F) der ICD-I0 "Psychische, Verhaltens- und Entwicklungsstörungen: Klinische Beschreibungen und diagnostische Leitlinien" (Fassung von 1987) wird in dieser Arbeit vorgestellt. Unter Berücksichtigung einiger für die Untersuchung in den deutschsprachigen Ländern vorgenommenen Modifikationen werden dabei die unterschiedlichen Phasen, die verwendeten Erhebungsinstrumente und die Stichprobendaten beschrieben.
large number of psychiatrists (Spitzer et al., 1979; Spitzer and Forman. 1979). At that time a DSM-III draft version was in use in many hospitals. First, 15 patients had to be rated in accordance with DSM-III before the field study was begun. In the field study itself each participant had to evaluate four patients which were compared across the United States. The chancecorrected agreement (Kappa; Cohen. 1960) for Axis I Disorders was 0.68 and 0.72 in two waves of data anaylsis (Spitzer et al., 1979). Burke (1988) notes that the most important question seems to be whether the raters really used the criteria of DSM-III, or whether they actually used other classifications. From the moment many used both systems in the same way, a possible mistake did not appear and therefore this variable is very hard to contro!. Study design oflCD-l 0 field trial The study was designed in the Mental Health Unit of WHO by a group of researchers who were assigned as advisers to the WHO. Another group, representatives of their countries during the development of ICD-I 0, were invited to participate in the study and to establish coordination centres. The methodology of the study was extensively discussed, and modified in accordance with a number of proposals. In the end, a total of 194 Field Trial Centres (FTC) in 54 countries coordinated by 17 Field Trial Coordinating Centres (FTCC) participated (Burke. 1988). For the German-speaking countries the psychiatrie clinic of the Lübeck Medical University was chosen to coordinate the participating centres in Austria, Switzerland, the German Democratic Republic and the Federal Republic of Germany. A committee for documentation and diagnoses had been set up in 1985 by the German Psychiatrie Association (DGPN). Since the chairman ofthis commission was the head
Downloaded by: NYU. Copyrighted material.
Summary
Design 01the ICD-J0 Field Trial in German-Speaking Countries of the Lübeck Psychiatrie University Department, our unit was asked to be the coordinating centre. Unfortunately, it was not possible for Psychiatrie Hospitals of the German Democratic Republic to take part in the study, possibly due to a lack ofmanpower and out of political reasons.
Pharmacopsychiatry 23 (1990) Teble 1 Phases of ICO-10 field trial Phase
1.
2. 3.
The WHO working group of coordinators met in Geneva on several occasions after various planning sessions; initial results on c1assification and guidelines were discussed in January '88.
4. 5. 6.
Stages 01the study design The design of the study involved several phases with a WHO-schedule and a specified timetable for the study in German-speaking countries (see table I):
143
7. 8. 9. 10.
WHO-Sehedule
Translation Fsmilisrization Preparation of ease summaries Joint assessment of pstients Assessment of esse summaries Further eomments and diseussion Preparation of preliminary synopsis by FTCC LUbeek Preparations of synopsis by FTCC LUbeek Oata analysis Preparation of papers
Timetable in Germanspeaking eountries
Preliminary Weeks 1-2
06.07. -19.07.1987
Weeks3-6
20.07.-16.08.1987
Weeks7-16
17.08. -25.10.1987
Weeks 17-18
26.10. -8.11.1987
Weeks 19-20
09.11-22.11.1987 23.11.-20.12.1987
-01.06.1988 June-Oet. 1988 Nov.-Oee.1988
-01.12.1988 -01.04.1989
A preliminary basic step was the translation of the English text of the "Clinieal Descriptions and Diagnostic Guidelines", draft 1987, into German. The preliminary translation was done by H. Dilling. K. Dilling, V. Dittmann and H. J. Freyberger (Klinik für Psychiatrie der Medizinischen Universität Lübeck; sections F I, F2, F3, F4), W. Mombaur. M. Zaudig. J. Mitte/hammer, W. Hil/er and R. Rummler (Poliklinik, Max-Planck-Institut für Psychiatrie iin München; sections FO, F5, F6, F7) and J. Niemeyer. K. Quaschner. H. Remschmidt and M H. Schmidt (Klinik für Kinder- und Jugendpsychiatrie, Zentralinstitut für Seelische Gesundheit, Mannheim; sections F8, F9). This provisional version was made available to the participating centres. An extract of three-character categories (FOO-F99) of this version is added in appendix 11.
Familiarization (phase 2) During the first part of the phase of familiarization, 5- 10 patients were to be examined and rated together. During the first part of this phase at least 5 patients had to be rated and diagnosed independently be each clinician. Discrepencies between the results then had to be discussed. These patients were to be weil known to the participating clinicians and their case histories already available. lt was also proposed to include both typical and diagnostieally diflicult cases. A short summary of the experience gained du ring the familiarization phase was to be sent to Geneva. In this way each centre had to send five case summaries and areport to the regional field trial coordinating centre. In the German-speaking countries the design of the familiarization phase was modified. During familiarization five written case summaries edited by the field trial coordinating centre Lübeck were assessed by participating clinicians.
Preparation of case summaries (phase 3) During weeks 3 to 6 each institution was asked to prepare four case summaries including two typieal cases and two cases associated with diagnostic difficulties. Only those cases familiar to the diagnosticians and those documented in a satisfactory manner were to be selected. In a later phase (assessment of case summaries) these case summaries were sent to the various hospitals participating in the study. Detailed guidelines for the preparation of the written case summaries were provided by the WHO, and included the following headings: Patient identification; present problem; family history; personal and social history; previous physieal iIIness; previous psychiatric contacts, assessments and iIInesses; previous (pre-morbid) personality; present
physieal examination; history of present mental iIIness; present mental state.
Joint assessment of patients (phase 4) During weeks 7 to 16 at least lO unselected patients had to be examined and evaluated. This could be done in two different ways: either by way of joint interviews conducted by two clinicians working together, or by inter changing the roles of interviewer and observer. The clinicians were required to fill in the diagnosis and comment form (the core instrument) independently of each other. No standardized interview was recommended, but it was expected that all interviews would include a general diagnostic comment whieh related not only to chapter V (F) ofICD-IO. Apart from this there was to be a history of the present episode, personal, medical and psychiatrie histories, as the family history and mental status. The interview was to take approximatly 45 minutes. The patient was examined in a case conference by one clinician. During this examination a complete diagnostic research took place and all material known about the patient was presented to the other participants. All clinicians were then asked to give their own independent diagnostic assessments. After completing the diagnoses and comment forms, any other considerations could be discussed, but the diagnoses given were not be altered. Following discussion of the case, the diagnostic consent of the group had to be recorded in writing. At the and of the joint assessment phase, diagnoses and comment forms from each clinician for at least 10 patients were to be submitted to data processing.
Assessment of case summaries (phase 5) Between weeks 17 and 18, the case summaries collected and distributed again by the FfCC were to be assessed by all partieipants in all the centres. Each institution received at least 10 case summaries, all of which had to be filled in by each clinician. At the end of this phase, 10 diagnoses and comment forms from each clinician participating were to be sent to the respective coordinating centre.
Further comments and discussion (phase 6) Between weeks 19 and 20, the clinicians were to write down their experience during the field study and refer to problems, special questions or changes of the ICD-IO chapter. All these comment and suggestion forms were to be sent to the coordinating centre.
Downloaded by: NYU. Copyrighted material.
Translation (phase I)
Pharmacopsychiatry 23 (1990)
H. Difljng. H. J. Freyberger. C. P. Malchow
reble 2 Partlcipating Centres, Number of Clinicians and Ratings
reble 3 Profession of the Participating Clinicians
Centres
Profession
Clinicians
Ratings
Adult Psychiatry: Klinik fUr Psychiatrie der Medizinischen Universität LUbeck (FTCC) Psychiatrische Universitätsklinik Basel Psychiatrische Klinik der Freien Universität Berlin Abteilung fUr Gerontopsychiatrie der Freien Universität Berlin Klinik fUr allgemeine Psychiatrie der Rheinischen- und Hochschulklinik Essen Bezirkskrankenhaus Kaufbeuren Psychiatrische Klinik der Universität Mainz Poliklinik des Max-Planck-Instituts fUr Psychiatrie, MUnchen Psychiatrische Klinik der Luclwig-Maximilians-Universität MUnchen Psychiatrische Universitätsklinik Wien Total
28 2
266
31
583
11
212
8 27 7
190 79
6
21
12 2
175
134
1,778
28
80
%
36 80
Senior Psychiatrists Psychiatrists in training Senior Psychologists Other Total
16 2
26.9 59.7 11.9 1.5
134
100.0
reble 4 Clincian's Familiarity with ICD-9 and DSM-III Degree of Familiarity Extensive Use Some Use Little or none
DSM-III
ICD-9
n
%
98
73.1 21.6
29 7
05.2
n
%
24
17.9
65 45
48.5 33.6
44 reble 5 Distribution of the Diagnostic Assessments in Different Phases of Data Evaluation
Child Psychiatry: Klinik fUr Kinder- und Jugendpsychiatrie, Zentralinstitut fUr Seelische Gesundheit, Mannheim Abt. fUr Kinder- und Jugendpsychiatrie, Max-Planck-Institut fUr Psychiatrie, MUnchen
n
4
160
3
120
The coordinating centres had the task of collecting the data provided by the institutions of their region, and of analysing important characteristics of the new ICD. The analysis is alleviated by assessing feasibility, suitability (goodness of fit) and reliability. This preparation 0/ a preliminary and afinal synopsis by FTCC Lübeck had to be finished by June 1988. After preparation of a global paper by the WHO, all the FTCC and FTC would have to publish their own reports.
Instruments For the evaluation different forrns edited by the WHO were used: (i) Study description form to record names and locations of participating field trial centres, names of the chief collaborating investigators and descriptions of the setting in which diagnostic judgements were made. (ii) Clinician identificationform to record name, age, sex, place of work, profession and three simple subjective ratings (familiarity with ICD-9; familiarity with DSM-III; Iikely use ofICD-IO) of each c1inician participating. For the field trial in Gerrnan-speaking countries the item "professional experience in psychiatry (in years)" was added (see appendix III: table I). (iii) Diagnosis and comment form to record the c1inician involved (including name and place of work), the case (incIuding the patient's name or code name, series, age, sex, date of evaluation and code for procedure), diagnoses (incIuding main, subsidiary and alternative diagnoses), goodness of fit rating, confidence rating, ease of difficulty of diagnosis rating, adequacy of c1inical descriptions and
Phase
n
Familiarization (phase 2) Joint assessment (phase 4) Case summaries (phase 5)
448 1,080
14.1 25.2 60.7
Total
1,778
100.0
250
%
diagnostic guidelines rating (see appendix III: table 2). For the field trial in Gerrnan-speaking countries a diagnostic assessment with ICD-9 and DSM-III was added. (iv) Comments and suggestions form to record general comments on Chapter V (F) as a whole, quality of translation and comments on each major section. Database
Participating centres andclinicians In the Gerrnan-speaking countries 10 centres with 134 c1inicians took part in the study (see table 2 and appendix 11: List of field trial centres and participating c1inici· ans). Some 1,778 diagnostic assessments of90 cases were provided. Aseparate study of child psychiatry with 280 diagnostic assessments was managed by the departments of child psychiatry of the Central Institute of Mental Health in Mannheim and the Max-Planck-Institute for Psychiatry in Munich. The majority of the c1inicians (50 females, 84 males aged between 26 and 58, mean - 35.6, sdev - 6.8) were psychiatrists in training (59.7%) (see table 3). The mean ofprofessional psychiatric experience (in years) was 5.2 (median - 3, sdev - 3.7). Concerning familiarity with the diagnostic system, 73.1 % of the c1inicians specified extensive use of ICD-9 and only 17.9% extensive use of DSM-III. Some 33.6%
Downloaded by: NYU. Copyrighted material.
144
Design 0/the ICD-I 0 Field Trial in German-Speaking Countries Teble 8 Proeedure of the Diagnostic Evaluation
Procedure
n
Live interview Video tape Written ease summary Written ease summary and live Interview or video tape
89 88 1,432
5.0 4.9 80.6
169
9.5
Total
1,778
100.0
%
Pharmacopsychiatry 23 (1990)
145
psychopathological symptoms, and on course-related criteria. Some artifical discrepancies in reliability and validity might be produced by the hierarchy of c1inical diagnoses (main vs. alterntive diagnoses) laid down by the design of this trial. Furthermore, the rating scales (goodness of fit; confidence of main diagnoses; ease or difficulty of main diagnoses; adequacy of c1inical descriptions and diagnostic guidelines) did not appear to be sufficiently differentiated, and statistically proved to be not independent (cf. Freyberger et al. 1990).
showed a low level of familiarity with DSM-III, but only 5.2 % little familiarity with ICD-9 (see table 4).
Patients and diagnostic assessments The majority of the 1,778 diagnostic assessments of90 patients (55 % female, 35 % male; mean age = 29.8 years, sdev - 22.2) were provided in phase V of the study (see table 5, cf. Stieglitz et al. 1990). Most of the ratings were given in the written case summaries (see table 6); only 5.0% in live interviews and 4.9% in video tapes. The distributions of diagnostic categories for main and subsidiary diagnoses are shown in appendix IV. For descriptive statistics (e. g. "goodness of fit" of different diagnostic categories) the total number of 1,778 assessments was applied; for the caIculation of percentge and chance-corrected agreement coefficients, data were confined to cases with at least 14 diagnostic assessments. After this empirical reduction, 31 cases with 1,475 ratings remained.
Criticism 0/thejield trialdesign The new international c1assification ICD-IO will be submitted for endorsement by the World Health Assembly in 1990. The field trial designed to produce an empirical"feedback" therefore had to be realized in a very short time. Familiarization with several new features incorporated in ICD-I 0 (cf. Dittmann and Dilling 1990) proved difficult for the participating c1inicians, and had an influence on the assessment ofthe results ofthis study. Therefore, a number of additional methodological aspects concerning the discussion of the results and the comparison of our findings with the resuIts of the field trial with the draft version of DSM-III (Spitzer et al., 1979) need to be considered. In our trial, validity and reliability depended on the quality ofthe case summaries and the given descriptions of
American Psychiatrie Association: Diagnostic and statistical manual of mental disorders (3rd edition) (DSM-III). APA, Washington D. C. 1980 American Psychiatrie Association: Diagnostic and statistical manual of mental disorders (3rd edition, revised) (DSM-IIIR). APA, Washington D. C. 1987 Burke, 1. D.: Field trials of the 1987 draft of chapter V (F) of ICD-I O. BriU. Psychiatry I52 (suppl. 1)(1988) 33- 37 Cohen, J.: A coefficient of agreement of nominal scales. Educational and Psychological Measurement 20 (1960) 37-46 Dittmann, v., H. Dilling: Chapter V (F) of ICD-10: Mental, behavioural and developmental disorders - introduction and overview. Pharmacopsychiatry Suppl. IV (1990) Freyberger, H. J., M. Albus, R. D. Slieglit::: ICD-IO field trial in German-speaking countries: Summary of the quantitative empirical results. Pharmacopsychiatry Suppl. IV (1990) Spitzer, R. L., J. B. W. Forman, J. Nee: DSM-III field trials: I. Initial interrater diagnostic reliability. Am. J. Psychiatry 136 (1979) 815817 Spitzer, R. L., 1. B. W. Forman: DSM-III field trials: 11. Initial experience with the multiaxial system. Am. J. Psychiatry 136 (1979) 818-820 Stieglitz, R. D., M. Zaudig, H. J. Freyberger, V. Dittmann: Interraterreliability, feasibility and suitability of ICD-IO during different stages ofthe field trial. Pharmacopsychiatry Suppl. IV (1990) Warld Health Organi::ation: Manual of the international statistical classification of diseases, injuries and causes of death, vols. land II. (ninth revision). World Health Organization, Geneva 1977 Warld Health Organizatian: ICD-IO, 1987 Draft ofChapter V, Categories FOO-F99, Mental, Behavioural and Developmental Disorders. Clinical Descriptions and Diagnostic Guidelines. Wotld Health Organization, Division of Mental Health, Geneva 1987 (MNH/MEP/87.I Rev. I) Warld Health Organizalian: ICD-I 0, 1988 Draft of chapter V, Categories FOO-F99, Mental, Behavioural and Developmental Disorders. C1inical Descriptions and Diagnostic Guidelines. World Health Organization, Division of Mental Health, Geneva 1987 (MNH/MEP/87.2 Rev. 2).
Prof Dr. H. Dilling
Klinik für Psychiatrie Med. Universität Lübeck Ratzeburger Allee 160 D-2400 Lübeck
Downloaded by: NYU. Copyrighted material.
References
Design of the ICD-IO Field Trial in German-Speaking Countries H. Dilling. H. J. Freyberger. C. P. Malchow Departrnent ofPsychiatry, (Director: Prof. Dr. H. Dilling), Lübeck Medical University, Lübeck, FRG
The study design of the international multicentric field trial on ICD-I0, chapter V (F) "Mental, Behavioural and Developmental Disorders: Clinical Descriptions and Diagnostic Guidelines" (1987 draft) including some modifications for the study in German-speaking countries, is described. The various stages of the field trial, instruments used, and the data base, are presented.
Introduction A preliminary version of ICD-l 0 chapter V (F) ("Mental, Behavioural and Developmental Disorders") has existed since 1984 and has undergone several modifications. In 1987/88 a field study was proposed by the Mental Health Division ofthe World Health Organization in Geneva with the aim of gaining experience with the current version (1987 draft; WHO, 1987) of the "Clinical Descriptions and Diagnostic Guidelines" of this chapter. The three major objectives of the field trial were (i) Assessment of the ease of understanding and ease of use of the new classification ("feasibility"). (ii) Assessment of the extent to which the new system adequately describes the disorders of various types of psychiatrie patients in routine clinical practice ("suitability, goodness offit"). And (iii) Assessment of the extent and type of agreement achieved between independent diagnosticians in different centres ("inter-rater reliability"). These aims should be accomplished by a large international study in 54 countries with different psychiatrie schools, so as to compare the classification under different social conditions and in different languages. The WHO was helped by the experience of the American Psychiatrie Association dating back to 1977, when during a two-year period DSM-III (APA, 1980) was tested by a
Pharmacopsychiatry 23 (1990) 142-145 (Supplement) C Georg Thieme Verlag Stuttgart·New York
Untersuchungsplan der ICD-I6-Feldstudie In dendeutschspncbigen Lindern Der Untersuchungsplan der internationalen multizentrischen Felduntersuchung zur Einführung des Kapitels V (F) der ICD-I0 "Psychische, Verhaltens- und Entwicklungsstörungen: Klinische Beschreibungen und diagnostische Leitlinien" (Fassung von 1987) wird in dieser Arbeit vorgestellt. Unter Berücksichtigung einiger für die Untersuchung in den deutschsprachigen Ländern vorgenommenen Modifikationen werden dabei die unterschiedlichen Phasen, die verwendeten Erhebungsinstrumente und die Stichprobendaten beschrieben.
large number of psychiatrists (Spitzer et al., 1979; Spitzer and Forman. 1979). At that time a DSM-III draft version was in use in many hospitals. First, 15 patients had to be rated in accordance with DSM-III before the field study was begun. In the field study itself each participant had to evaluate four patients which were compared across the United States. The chancecorrected agreement (Kappa; Cohen. 1960) for Axis I Disorders was 0.68 and 0.72 in two waves of data anaylsis (Spitzer et al., 1979). Burke (1988) notes that the most important question seems to be whether the raters really used the criteria of DSM-III, or whether they actually used other classifications. From the moment many used both systems in the same way, a possible mistake did not appear and therefore this variable is very hard to contro!. Study design oflCD-l 0 field trial The study was designed in the Mental Health Unit of WHO by a group of researchers who were assigned as advisers to the WHO. Another group, representatives of their countries during the development of ICD-I 0, were invited to participate in the study and to establish coordination centres. The methodology of the study was extensively discussed, and modified in accordance with a number of proposals. In the end, a total of 194 Field Trial Centres (FTC) in 54 countries coordinated by 17 Field Trial Coordinating Centres (FTCC) participated (Burke. 1988). For the German-speaking countries the psychiatrie clinic of the Lübeck Medical University was chosen to coordinate the participating centres in Austria, Switzerland, the German Democratic Republic and the Federal Republic of Germany. A committee for documentation and diagnoses had been set up in 1985 by the German Psychiatrie Association (DGPN). Since the chairman ofthis commission was the head
Downloaded by: NYU. Copyrighted material.
Summary
Design 01the ICD-J0 Field Trial in German-Speaking Countries of the Lübeck Psychiatrie University Department, our unit was asked to be the coordinating centre. Unfortunately, it was not possible for Psychiatrie Hospitals of the German Democratic Republic to take part in the study, possibly due to a lack ofmanpower and out of political reasons.
Pharmacopsychiatry 23 (1990) Teble 1 Phases of ICO-10 field trial Phase
1.
2. 3.
The WHO working group of coordinators met in Geneva on several occasions after various planning sessions; initial results on c1assification and guidelines were discussed in January '88.
4. 5. 6.
Stages 01the study design The design of the study involved several phases with a WHO-schedule and a specified timetable for the study in German-speaking countries (see table I):
143
7. 8. 9. 10.
WHO-Sehedule
Translation Fsmilisrization Preparation of ease summaries Joint assessment of pstients Assessment of esse summaries Further eomments and diseussion Preparation of preliminary synopsis by FTCC LUbeek Preparations of synopsis by FTCC LUbeek Oata analysis Preparation of papers
Timetable in Germanspeaking eountries
Preliminary Weeks 1-2
06.07. -19.07.1987
Weeks3-6
20.07.-16.08.1987
Weeks7-16
17.08. -25.10.1987
Weeks 17-18
26.10. -8.11.1987
Weeks 19-20
09.11-22.11.1987 23.11.-20.12.1987
-01.06.1988 June-Oet. 1988 Nov.-Oee.1988
-01.12.1988 -01.04.1989
A preliminary basic step was the translation of the English text of the "Clinieal Descriptions and Diagnostic Guidelines", draft 1987, into German. The preliminary translation was done by H. Dilling. K. Dilling, V. Dittmann and H. J. Freyberger (Klinik für Psychiatrie der Medizinischen Universität Lübeck; sections F I, F2, F3, F4), W. Mombaur. M. Zaudig. J. Mitte/hammer, W. Hil/er and R. Rummler (Poliklinik, Max-Planck-Institut für Psychiatrie iin München; sections FO, F5, F6, F7) and J. Niemeyer. K. Quaschner. H. Remschmidt and M H. Schmidt (Klinik für Kinder- und Jugendpsychiatrie, Zentralinstitut für Seelische Gesundheit, Mannheim; sections F8, F9). This provisional version was made available to the participating centres. An extract of three-character categories (FOO-F99) of this version is added in appendix 11.
Familiarization (phase 2) During the first part of the phase of familiarization, 5- 10 patients were to be examined and rated together. During the first part of this phase at least 5 patients had to be rated and diagnosed independently be each clinician. Discrepencies between the results then had to be discussed. These patients were to be weil known to the participating clinicians and their case histories already available. lt was also proposed to include both typical and diagnostieally diflicult cases. A short summary of the experience gained du ring the familiarization phase was to be sent to Geneva. In this way each centre had to send five case summaries and areport to the regional field trial coordinating centre. In the German-speaking countries the design of the familiarization phase was modified. During familiarization five written case summaries edited by the field trial coordinating centre Lübeck were assessed by participating clinicians.
Preparation of case summaries (phase 3) During weeks 3 to 6 each institution was asked to prepare four case summaries including two typieal cases and two cases associated with diagnostic difficulties. Only those cases familiar to the diagnosticians and those documented in a satisfactory manner were to be selected. In a later phase (assessment of case summaries) these case summaries were sent to the various hospitals participating in the study. Detailed guidelines for the preparation of the written case summaries were provided by the WHO, and included the following headings: Patient identification; present problem; family history; personal and social history; previous physieal iIIness; previous psychiatric contacts, assessments and iIInesses; previous (pre-morbid) personality; present
physieal examination; history of present mental iIIness; present mental state.
Joint assessment of patients (phase 4) During weeks 7 to 16 at least lO unselected patients had to be examined and evaluated. This could be done in two different ways: either by way of joint interviews conducted by two clinicians working together, or by inter changing the roles of interviewer and observer. The clinicians were required to fill in the diagnosis and comment form (the core instrument) independently of each other. No standardized interview was recommended, but it was expected that all interviews would include a general diagnostic comment whieh related not only to chapter V (F) ofICD-IO. Apart from this there was to be a history of the present episode, personal, medical and psychiatrie histories, as the family history and mental status. The interview was to take approximatly 45 minutes. The patient was examined in a case conference by one clinician. During this examination a complete diagnostic research took place and all material known about the patient was presented to the other participants. All clinicians were then asked to give their own independent diagnostic assessments. After completing the diagnoses and comment forms, any other considerations could be discussed, but the diagnoses given were not be altered. Following discussion of the case, the diagnostic consent of the group had to be recorded in writing. At the and of the joint assessment phase, diagnoses and comment forms from each clinician for at least 10 patients were to be submitted to data processing.
Assessment of case summaries (phase 5) Between weeks 17 and 18, the case summaries collected and distributed again by the FfCC were to be assessed by all partieipants in all the centres. Each institution received at least 10 case summaries, all of which had to be filled in by each clinician. At the end of this phase, 10 diagnoses and comment forms from each clinician participating were to be sent to the respective coordinating centre.
Further comments and discussion (phase 6) Between weeks 19 and 20, the clinicians were to write down their experience during the field study and refer to problems, special questions or changes of the ICD-IO chapter. All these comment and suggestion forms were to be sent to the coordinating centre.
Downloaded by: NYU. Copyrighted material.
Translation (phase I)
Pharmacopsychiatry 23 (1990)
H. Difljng. H. J. Freyberger. C. P. Malchow
reble 2 Partlcipating Centres, Number of Clinicians and Ratings
reble 3 Profession of the Participating Clinicians
Centres
Profession
Clinicians
Ratings
Adult Psychiatry: Klinik fUr Psychiatrie der Medizinischen Universität LUbeck (FTCC) Psychiatrische Universitätsklinik Basel Psychiatrische Klinik der Freien Universität Berlin Abteilung fUr Gerontopsychiatrie der Freien Universität Berlin Klinik fUr allgemeine Psychiatrie der Rheinischen- und Hochschulklinik Essen Bezirkskrankenhaus Kaufbeuren Psychiatrische Klinik der Universität Mainz Poliklinik des Max-Planck-Instituts fUr Psychiatrie, MUnchen Psychiatrische Klinik der Luclwig-Maximilians-Universität MUnchen Psychiatrische Universitätsklinik Wien Total
28 2
266
31
583
11
212
8 27 7
190 79
6
21
12 2
175
134
1,778
28
80
%
36 80
Senior Psychiatrists Psychiatrists in training Senior Psychologists Other Total
16 2
26.9 59.7 11.9 1.5
134
100.0
reble 4 Clincian's Familiarity with ICD-9 and DSM-III Degree of Familiarity Extensive Use Some Use Little or none
DSM-III
ICD-9
n
%
98
73.1 21.6
29 7
05.2
n
%
24
17.9
65 45
48.5 33.6
44 reble 5 Distribution of the Diagnostic Assessments in Different Phases of Data Evaluation
Child Psychiatry: Klinik fUr Kinder- und Jugendpsychiatrie, Zentralinstitut fUr Seelische Gesundheit, Mannheim Abt. fUr Kinder- und Jugendpsychiatrie, Max-Planck-Institut fUr Psychiatrie, MUnchen
n
4
160
3
120
The coordinating centres had the task of collecting the data provided by the institutions of their region, and of analysing important characteristics of the new ICD. The analysis is alleviated by assessing feasibility, suitability (goodness of fit) and reliability. This preparation 0/ a preliminary and afinal synopsis by FTCC Lübeck had to be finished by June 1988. After preparation of a global paper by the WHO, all the FTCC and FTC would have to publish their own reports.
Instruments For the evaluation different forrns edited by the WHO were used: (i) Study description form to record names and locations of participating field trial centres, names of the chief collaborating investigators and descriptions of the setting in which diagnostic judgements were made. (ii) Clinician identificationform to record name, age, sex, place of work, profession and three simple subjective ratings (familiarity with ICD-9; familiarity with DSM-III; Iikely use ofICD-IO) of each c1inician participating. For the field trial in Gerrnan-speaking countries the item "professional experience in psychiatry (in years)" was added (see appendix III: table I). (iii) Diagnosis and comment form to record the c1inician involved (including name and place of work), the case (incIuding the patient's name or code name, series, age, sex, date of evaluation and code for procedure), diagnoses (incIuding main, subsidiary and alternative diagnoses), goodness of fit rating, confidence rating, ease of difficulty of diagnosis rating, adequacy of c1inical descriptions and
Phase
n
Familiarization (phase 2) Joint assessment (phase 4) Case summaries (phase 5)
448 1,080
14.1 25.2 60.7
Total
1,778
100.0
250
%
diagnostic guidelines rating (see appendix III: table 2). For the field trial in Gerrnan-speaking countries a diagnostic assessment with ICD-9 and DSM-III was added. (iv) Comments and suggestions form to record general comments on Chapter V (F) as a whole, quality of translation and comments on each major section. Database
Participating centres andclinicians In the Gerrnan-speaking countries 10 centres with 134 c1inicians took part in the study (see table 2 and appendix 11: List of field trial centres and participating c1inici· ans). Some 1,778 diagnostic assessments of90 cases were provided. Aseparate study of child psychiatry with 280 diagnostic assessments was managed by the departments of child psychiatry of the Central Institute of Mental Health in Mannheim and the Max-Planck-Institute for Psychiatry in Munich. The majority of the c1inicians (50 females, 84 males aged between 26 and 58, mean - 35.6, sdev - 6.8) were psychiatrists in training (59.7%) (see table 3). The mean ofprofessional psychiatric experience (in years) was 5.2 (median - 3, sdev - 3.7). Concerning familiarity with the diagnostic system, 73.1 % of the c1inicians specified extensive use of ICD-9 and only 17.9% extensive use of DSM-III. Some 33.6%
Downloaded by: NYU. Copyrighted material.
144
Design 0/the ICD-I 0 Field Trial in German-Speaking Countries Teble 8 Proeedure of the Diagnostic Evaluation
Procedure
n
Live interview Video tape Written ease summary Written ease summary and live Interview or video tape
89 88 1,432
5.0 4.9 80.6
169
9.5
Total
1,778
100.0
%
Pharmacopsychiatry 23 (1990)
145
psychopathological symptoms, and on course-related criteria. Some artifical discrepancies in reliability and validity might be produced by the hierarchy of c1inical diagnoses (main vs. alterntive diagnoses) laid down by the design of this trial. Furthermore, the rating scales (goodness of fit; confidence of main diagnoses; ease or difficulty of main diagnoses; adequacy of c1inical descriptions and diagnostic guidelines) did not appear to be sufficiently differentiated, and statistically proved to be not independent (cf. Freyberger et al. 1990).
showed a low level of familiarity with DSM-III, but only 5.2 % little familiarity with ICD-9 (see table 4).
Patients and diagnostic assessments The majority of the 1,778 diagnostic assessments of90 patients (55 % female, 35 % male; mean age = 29.8 years, sdev - 22.2) were provided in phase V of the study (see table 5, cf. Stieglitz et al. 1990). Most of the ratings were given in the written case summaries (see table 6); only 5.0% in live interviews and 4.9% in video tapes. The distributions of diagnostic categories for main and subsidiary diagnoses are shown in appendix IV. For descriptive statistics (e. g. "goodness of fit" of different diagnostic categories) the total number of 1,778 assessments was applied; for the caIculation of percentge and chance-corrected agreement coefficients, data were confined to cases with at least 14 diagnostic assessments. After this empirical reduction, 31 cases with 1,475 ratings remained.
Criticism 0/thejield trialdesign The new international c1assification ICD-IO will be submitted for endorsement by the World Health Assembly in 1990. The field trial designed to produce an empirical"feedback" therefore had to be realized in a very short time. Familiarization with several new features incorporated in ICD-I 0 (cf. Dittmann and Dilling 1990) proved difficult for the participating c1inicians, and had an influence on the assessment ofthe results ofthis study. Therefore, a number of additional methodological aspects concerning the discussion of the results and the comparison of our findings with the resuIts of the field trial with the draft version of DSM-III (Spitzer et al., 1979) need to be considered. In our trial, validity and reliability depended on the quality ofthe case summaries and the given descriptions of
American Psychiatrie Association: Diagnostic and statistical manual of mental disorders (3rd edition) (DSM-III). APA, Washington D. C. 1980 American Psychiatrie Association: Diagnostic and statistical manual of mental disorders (3rd edition, revised) (DSM-IIIR). APA, Washington D. C. 1987 Burke, 1. D.: Field trials of the 1987 draft of chapter V (F) of ICD-I O. BriU. Psychiatry I52 (suppl. 1)(1988) 33- 37 Cohen, J.: A coefficient of agreement of nominal scales. Educational and Psychological Measurement 20 (1960) 37-46 Dittmann, v., H. Dilling: Chapter V (F) of ICD-10: Mental, behavioural and developmental disorders - introduction and overview. Pharmacopsychiatry Suppl. IV (1990) Freyberger, H. J., M. Albus, R. D. Slieglit::: ICD-IO field trial in German-speaking countries: Summary of the quantitative empirical results. Pharmacopsychiatry Suppl. IV (1990) Spitzer, R. L., J. B. W. Forman, J. Nee: DSM-III field trials: I. Initial interrater diagnostic reliability. Am. J. Psychiatry 136 (1979) 815817 Spitzer, R. L., 1. B. W. Forman: DSM-III field trials: 11. Initial experience with the multiaxial system. Am. J. Psychiatry 136 (1979) 818-820 Stieglitz, R. D., M. Zaudig, H. J. Freyberger, V. Dittmann: Interraterreliability, feasibility and suitability of ICD-IO during different stages ofthe field trial. Pharmacopsychiatry Suppl. IV (1990) Warld Health Organi::ation: Manual of the international statistical classification of diseases, injuries and causes of death, vols. land II. (ninth revision). World Health Organization, Geneva 1977 Warld Health Organizatian: ICD-IO, 1987 Draft ofChapter V, Categories FOO-F99, Mental, Behavioural and Developmental Disorders. Clinical Descriptions and Diagnostic Guidelines. Wotld Health Organization, Division of Mental Health, Geneva 1987 (MNH/MEP/87.I Rev. I) Warld Health Organizalian: ICD-I 0, 1988 Draft of chapter V, Categories FOO-F99, Mental, Behavioural and Developmental Disorders. C1inical Descriptions and Diagnostic Guidelines. World Health Organization, Division of Mental Health, Geneva 1987 (MNH/MEP/87.2 Rev. 2).
Prof Dr. H. Dilling
Klinik für Psychiatrie Med. Universität Lübeck Ratzeburger Allee 160 D-2400 Lübeck
Downloaded by: NYU. Copyrighted material.
References
