Development of a Scale for the Measurement Symptom Change in an Outpatient Clinic
of
Robert Plutchik, Hope R. Conte, Warren Spence, Peter Buckley, and Toksoz B. Karasu A brief 21-item symptom rating scale, the Psychiatric Outpatient Rating Scale (PORS), was developed for use in outpatient clinics. On the basis of its initial use with 86 patients, it was shown to have high internal and interjudge reliability and evidence of concurrent and construct validity. Scores on the PORS correlated significantly with the Global Assessment Scale and with the number of sessions of psychotherapy. For a subsample of 45 patients rated on the PORS at the beginning and termination of psychotherapy, seven symptoms revealed highly significant improvement. The PORS appears to be a potentially useful measure of change in outpatient clinics. 0 1990 by W. B. Saunders Company.
A
NUMBER OF EVALUATION INSTRUMENTS exist for measuring changes that occur in psychiatric patients during treatment. In addition to projective tests and self-report scales, a number of psychiatric rating scales exist.’ Each method has both strengths and weaknesses. For example, although projective tests are less liable to faking, they suffer from limited reliability and standardization. Results of self-report scales are often subject to defensiveness and denial, but they are sometimes the best avenue to the patient’s inner feelings. Rating scales generally require highly trained raters, but they are less dependent on the occasional idiosyncratic verbalizations of the patients. In addition, they can provide information on a patient even if the patient unexpectedly drops out of treatment and is unavailable for follow-up testing. This last point is important in connection with the use of testing procedures in outpatient clinics. Most patients who come for outpatient psychiatric treatment tend to stay for relatively few sessions. In the university-affiliated, municipal hospital clinic where the present study was done, the average number of sessions attended is about six, which is typical of many other such clinics. Many patients, without notification, simply stop coming to the clinic after a few visits. The result is that efforts to measure change associated with clinic treatment in such cases must be based on psychiatric ratings rather than self-reports. Of the rating scales in current use, some are designed for measuring single dimensions such as depression2 Others, such as the Brief Psychiatric Rating Scale (BPRS)3 are designed for inpatients and thus do not cover the broad range of problems encountered in an outpatient clinic. The Global Assessment Scale (GAS)4 is designed to evaluate overall functioning on a single loo-point scale. One disadvantage of the GAS, however, is that the overall numerical rating does not identify the separate components that enter into life-adjustment and psychopathology. Another disadvantage is that short-term psychotherapy seldom produces much change in overall functioning. Thus, a need still exist for a rating instrument that
From the Albert Einstein College of Medicine/Monfefiore Medical Center, Bronx, NY. Address reprint requests to Robert Plutckik. Ph.D., Bronx Municipal Hospital Center, Pelkam Parkway S. and Eastchester Rd., Bronx, NY 10461. o 1990 by W. B. Saunders Company. 0010-440X/90/3102-0003$03.00/0 134
Comprehensive
Psychiatry,
Vol. 3 1, No. 2 (March/April),
1990: pp 134-l 39
MEASUREMENT
OF SYMPTOM
CHANGE
135
can be used quickly and reliably and that covers the full range of problems typical of psychiatric clinic outpatients. We describe the development of the Psychiatric Outpatient Rating Scale (PORS), a rating instrument that can be completed by a therapist and that has several important features: (a) It is designed with the primary objective of measuring symptoms and problems often found in psychiatric clinic outpatients; (b) it is quick and relatively simple to complete and score; (c) it provides a global score as well as individual scores for symptoms or problems; and (d) it covers the most frequently encountered problems which outpatients bring to a clinic. METHOD The PORS consists of 21 items (Table 1). A clinician rates the level of severity of each category ranging from 0 (not present) to 1 (minimal problem), to 2 (moderate problem), to 3 (severe problem), to 4 (very severe problem). These ratings may be cumulated to provide a total score. Consistent with reports of other patient rating scales,5.’ explicit definitions were not provided for the terms used, although measures of interjudge reliability were obtained. Interjudge reliability was determined in the following way. For 12 different patients, a supervising psychiatrist sat in with the psychiatric resident or psychology intern during the initial interview. At the end of the session, the interviewing clinician and the supervisor separately rated the patient for symptoms and problems on the PORS. Eight different residents and three different supervisors were involved. Product-moment correlations were then obtained between clinician and supervisor for each patient across the 21 items of the PORS. These correlations ranged from approximately + 50 to + .95 with a mean interjudge correlation of + .74 (significant at a level of more than .Ol). Thus, the PORS items apparently can be reliably judged by a clinician after an initial interview. This finding is consistent with those of other reports. For example, interrater reliability for individual items of the BPRS average +.75.* Similarly, interjudge reliabilities for the items of the Psychiatric Status Schedule’ range from a low of +.57 to a high of +.98 (mean correlation + .87). Interjudge reliability of the General Adult Inpatient Psychiatric Assessment Scale’ was also comparable. During an
Table 1. Frequency
Item 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 12. 13. 14. 15. 16. 17. 18. 19. 20. 21.
Self-esteem Depression Anxiety Dependency Social isolation Job or school problem lmpulsivity Sleep disturbance Marital conflicts Aggressive outbursts Problems with children Somatization Suicidal impulses or behaviors Sexual dysfunction Substance abuse Delusions Formal thought disorder Panic attacks Hallucinations Obsessive-compulsive rituals Phobias
NOTE. N = 86.
of Occurrence
of Symptoms
in Percent
No Problem
Minimal Problem
Moderate Problem
Severe Problem
3.5 5.8 11.5 19.8 20.9 33.7 40.7 43.0 50.9 55.8 59.3 59.3 60.5 66.3 70.9 79.1 81.4 83.7 83.7 86.0 86.0
9.3 31.4 20.9 15.1 15.1 15.1 16.3 26.6 7.0 10.5 11.6 23.3 29.1 16.3 11.6 5.8 7.0 4.7 4.7 7.0 2.3
37.2 40.7 43.0 27.9 27.9 26.7 25.6 18.6 15.1 17.4 9.3 7.0 8.1 9.3 8.1 9.3 8.1 5.8 8.1 3.5 7.0
50.0 22.1 24.5 37.2 36.0 24.4 17.4 12.8 27.9 16.3 19.8 10.5 2.3 8.1 9.3 5.9 3.5 5.9 3.5 3.5 4.6
PLUTCHIK ET AL
136
interviewer-observer setting, interjudge agreement (K) for individual items ranged from +.61 to 1.00 (mean + .80). Thus, clinicians apparently can rate individual descriptive items reliably on psychiatric assessment scales. The PGRS was developed by a committee of five experienced clinicians who have had extensive experience in psychiatric outpatient clinics. Their mean level of postgraduate experience was 12 years 4 months. The format and terminology of the scales underwent three revisions after several pilot studies and discussion with the therapists who actually used the scale. The setting for the study was a large municipal hospital outpatient clinic that handles approximately 10,000 visits and 1,200 new patients each year. The therapists are third-year psychiatric residents and psychology interns under supervision, as well as experienced social workers. In this clinic, two thirds of the patients leave within 12 sessions. Most receive individual therapy, although many patients also receive group, couple, and family therapy. For many of the patients, antianxiety or antidepression medication is prescribed. The present study used a retrospective design, based on patient records. Charts of 86 consecutive patients for whom at least a PORS initial evaluation had been completed were selected for inclusion. Of these 86 charts, 45 also contained a PORS discharge evaluation. The charts were read by an independent evaluator who was instructed to make two ratings on the GAS, one based on the symptomatology presented at intake and the other based on the description of the patient in the discharge summary. The GAS consists of a loo-point rating scale with descriptions of functional level provided at every lo-point increment; high scores indicate better levels of life functioning. To establish the interjudge reliability of the GAS rating, two evaluators independently rated 10 records each. The correlation between the two raters was +.82. The remaining records were then assessed by a single evaluator.
RESULTS
Table 2 shows general demographic and personal information about the sample of 86 patients studied. Men and women were almost equally represented. The age range was 18 to 67 years (mean, 39 years; SD = 12.7). The median number of outpatient sessions was 12 (range, 1 to 61 sessions). At the time of discharge, 26% of the patients were diagnosed as personality disorders, 23% as affective disorders, 19% as schizophrenic disorders, 19% as adjustment disorders, and 9% as anxiety disorder. Four percent of the sample had no psychiatric diagnosis at the time of discharge. Table 2. Demographic
and Diagnostic
Characteristics
Sex M =41 F = 45 Age (vr) Mean = 39.00 SD = 12.7 Range = 16-67 No. of sessions attended Median = 12 Mean = 12 SD = 14.2 Range = l-6 1 DSM-III discharge diagnoses (%) Personality disorder (26) Affective disorder (23) Schizophrenic disorder (19) Adjustment disorder (19) Anxiety disorder (9) No diagnosis given (4) NOTE. N = 86.
of the Sample
MEASUREMENT
OF SYMPTOM
CHANGE
137
Table 1 shows the frequency of occurrence of problems at the time of admission to the outpatient clinic, as rated by the clinician. Problems were evaluated as having different levels of severity; e.g., depression was a minimal problem for 31% of the sample, a moderate problem for 41% of the sample, and a severe problem for 22% of the sample. (To simplify Table 1, ratings for severe and very severe were combined.) The most common severe and moderate problems concerned self-esteem, social isolation, dependency, anxiety, and depression. The ubiquity of these symptoms suggests that for most outpatients they represent core complaints or accompany a wide range of presenting problems, or both. Symptoms suggestive of more serious pathology, such as might be typically found in an inpatient service, were not frequently encountered in this sample. The relatively few patients rated as having a thought disorder or hallucinations reflect a subgroup of patients referred to the outpatient clinic for follow-up treatment and medication after a period of inpatient hospitalization. Measurement
of Change
Table 3 summarizes the changes that occurred in the patients during their stay in the clinic. Of the original sample, 45 of the diagnosed patients had an evaluation made on each of the 21 items of the PORS at both intake and discharge. Mean scores for each item were obtained, and the initial mean rating for each item was compared with the discharge mean rating by a t test for correlated data. The results showed a significant decrease in severity of the problems for 7 of the 21 items. There was a significant decrease in severity of depression, anxiety, job or school problems or both, feelings of social isolation, temper outbursts, sleep Table 3. Significance
Tests for Differences
Item 1. 2. 3. 4. 5. 6. 7. a. 9. 10. 11. 12. 13. 14. 15. 16. 17. la. 19. 20. 21.
Delusions Sleep disturbance Anxien/ Job or school problem Aggressive outbursts Social isolation Depression Self esteem Substance abuse Panic attacks Problems with children Formal thought disorder Marital conflicts lmpulsivity Phobias Suicidal impulses or behavior Sexual dysfunction Obsessive compulsive rituals Somatization Hallucinations Dependency
NOTE. N = 45. lP< .Ol tP < .05
Between
Initial and Discharge
Ratings
Initial Mean
Discharge Mean
Initial SD
Discharge SD
t
0.49 1.04 2.00 I .5a 0.78 2.00 I .a9 2.51 0.31 0.40 1.04 0.44 1.75 0.97 0.47 0.49 0.73 0.31 0.80 0.31 2.07
0.16 0.58 1.53 1.24 0.53 1.64 1.62 2.24 0.38 0.31 0.82 0.29 1.62 0.82 0.38 0.29 0.55 0.22 0.69 0.24 2.02
0.99 1.19 I .28 1.39 1.22 1.37 0.83 0.99 0.63 1.10 1.52 0.92 1.67 1.12 1.10 0.76 1.16 0.70 1.16 0.82 1.42
0.52 0.97 1.08 1.23 1.01 1.30 0.89 1.03 0.75 0.91 1.32 0.79 1.54 1.16 0.91 0.87 0.87 0.64 1.02 0.88 1.32
2.91* 2.75’ 2.43T 2.4lT 2.41t 2.33T 2.01t 1.81 ’ 1.78 1.43 1.43 1.42 1.35 1.31 1.27 1.24 1.16 1.16 0.93 0.44 0.36
138
PLUTCHIK ET AL
problems, and delusions. All problems except substance abuse, which apparently increased during the course of therapy, showed a decrease in severity, although not of a statistically significant magnitude. The average number of therapy sessions for the 45 patients was 21 (SD, 14.9). Internal Reliability of the PORS
Until now, we have analyzed each item separately. Whether the items rated by the clinician form a psychometric scale is questionable, however. To determine this issue, the internal reliability of the group of items as a whole was computed by means of coefficient alpha. Internal reliability for the initial sample of 86 patients was .70. For the subgroup of 45 patients who were evaluated initially and at the time of discharge, the coefficient alpha was .84. Thus, the items of the PORS rating scales appear to be fairly highly intercorrelated and the PORS apparently can be treated as a scale whose total is a meaningful reflection of the individual item responses. Concurrent Validity
A further analysis was performed to determine the relationship between a global assessment of the level of dysfunction of the patient and the total score on the PORS. The global assessment was made by the GAS. The GAS score was determined for 35 patients and then correlated against the PORS scores of the same patients. The correlation was -.64, significant at the .OOl level. (The minus sign reflects the fact that high scores on the GAS express good functioning whereas high scores on the PORS reflect poor functioning.) Construct Validity
The final analysis was the determination of the correlation between total PORS change scores and number of sessions. This was -.37 (P < .02), indicating that patients who remained in treatment tended to have greater decreases in problem severity. The same analysis was computed between the number of sessions and GAS change scores. The correlation +.40 (P -z .Ol) indicating again that the longer patients remained in treatment, the greater the increase in their overall functioning. DISCUSSION
We present data obtained on a group of outpatients using a new 21-item symptom rating scale, the PORS. The internal reliability of the PORS was high, its interjudge reliability was high, and its correlation with the well-known GAS was -.64, indicating a reasonable degree of concurrent validity. As a measure of construct validity, the PORS correlated .37 with number of sessions of psychotherapy, indicating that patients who stayed in treatment longer tended to improve more. This finding is consistent with the conclusion of Howard et al.,9 who reported a dose-effect relationship based on a metaanalysis of 15 clinical studies of psychotherapy. The percentage of improved patients increased rapidly up to about 26 sessions and then gradually leveled off. A roughly similar observation was made in the present study, although the treatments were quite diverse and included individual and group therapy as well as psychotropic medication. The population of this outpatient clinic is quite heterogeneous. For the most part, the patients are of a lower middle class social status and consider the clinic a place
MEASUREMENT
OF SYMPTOM
CHANGE
139
where they can try to solve problems in a relatively brief period. A certain proportion of the patients are former psychiatric inpatients who have been referred to the outpatient clinic for follow-up treatment. Such treatment consists in most cases of psychotropic medication and supportive individual or group sessions. Because of the heterogeneity of the population, some of the symptoms that improved significantly are worth considering. These are (in order of magnitude of change) delusions, sleep disturbance, anxiety, aggressive outbursts, and depression. Most of these problems tend to benefit the most from psychotropic medication and, considering the relatively short-term therapy received by these patients, the average changes observed may reflect the benefits of drugs as much as the benefits of psychotherapy. In contrast, there was relatively little change in such symptoms as dependency, obsessive compulsive behavior, impulsivity, and marital conflicts. This observation may reflect the fact that these symptoms represent personality traits or chronic tendencies, characteristics that are less affected by drugs and are influenced only by much therapeutic interaction. A similar point was made by Weissman et al.” We cannot specify exactly what proportion of improvement results from drugs and what proportion results from psychotherapy, although Conte et al.” have shown in a metaanalysis that some summation of drug plus psychotherapy effects does occur. The current findings are consistent with the possibility of a drug/ psychotherapy interaction. The PORS may be a useful tool for studying change in psychotherapy. It is simple and easy to use and covers a wide range of outpatient problems and symptoms. It may also serve to sensitize the clinician to consider issues that might otherwise be missed, thereby enhancing therapeutic effectiveness. REFERENCES 1. Plutchik R, Conte HR: Quantitative assessment of personality disorders, in Michels R, Cavenar JO Jr, Brodie HKH, et al (eds) Psychiatry, vol. 1, Philadelphia, Lippincott, 1985 2. Hamilton M: A rating scale for depression. J Nemo1 Neurosurg Psychiatry 2356-62, 1960 3. Overall JE, Gorham DR: The Brief Psychiatric Rating Scale. Psycho1 Rep 10:799-812, 1962 4. Endicott J, Spitzer RL, Fleiss JL, et al: The Global Assessment Scale. Arch Gen Psychiatry 331766771, 1976 5. Spitzer RL, Endicott J, Fleiss JL, et al: The Psychiatric Status Schedule: A technique for evaluating psychopathology and impairment in role functioning. Arch Gen Psychiatry 23:41-S& 1970 6. Overall JE, Hollister LE, Pichot P: Major psychiatric disorders: A four-dimensional model. Arch Gen Psychiatry 15:146-151, 1967 7. Summers WK, Marsh GM, Chiong B, et al: The General Adult Inpatient Psychiatric Assessment Scale (GAIPAS). Psychiatry Res 10:217-236, 1983 8. Hedlund JL, Vieweg BW: The Brief Psychiatric Rating Scale (BPRS): A comprehensive review. J Operational Psychiatry 11:48-65, 1980 9. Howard KI, Kopta SM, Krause MS, et al: The dose-effect relationship in psychotherapy. Am Psycho1 41:159-164,1986 10. Weissman MM, Klerman GL, Prusoff GA, et al: Depressed outpatients: One year after treatment with drugs and/or interpersonal psychotherapy. Arch Gen Psychiatry 38:51-55, 1981 11. Conte HR, Plutchik R, Wild KV, et al: Combined psychotherapy and pharmacotherapy for depression. Arch Gen Psychiatry 43:471-479, 1986
of
Robert Plutchik, Hope R. Conte, Warren Spence, Peter Buckley, and Toksoz B. Karasu A brief 21-item symptom rating scale, the Psychiatric Outpatient Rating Scale (PORS), was developed for use in outpatient clinics. On the basis of its initial use with 86 patients, it was shown to have high internal and interjudge reliability and evidence of concurrent and construct validity. Scores on the PORS correlated significantly with the Global Assessment Scale and with the number of sessions of psychotherapy. For a subsample of 45 patients rated on the PORS at the beginning and termination of psychotherapy, seven symptoms revealed highly significant improvement. The PORS appears to be a potentially useful measure of change in outpatient clinics. 0 1990 by W. B. Saunders Company.
A
NUMBER OF EVALUATION INSTRUMENTS exist for measuring changes that occur in psychiatric patients during treatment. In addition to projective tests and self-report scales, a number of psychiatric rating scales exist.’ Each method has both strengths and weaknesses. For example, although projective tests are less liable to faking, they suffer from limited reliability and standardization. Results of self-report scales are often subject to defensiveness and denial, but they are sometimes the best avenue to the patient’s inner feelings. Rating scales generally require highly trained raters, but they are less dependent on the occasional idiosyncratic verbalizations of the patients. In addition, they can provide information on a patient even if the patient unexpectedly drops out of treatment and is unavailable for follow-up testing. This last point is important in connection with the use of testing procedures in outpatient clinics. Most patients who come for outpatient psychiatric treatment tend to stay for relatively few sessions. In the university-affiliated, municipal hospital clinic where the present study was done, the average number of sessions attended is about six, which is typical of many other such clinics. Many patients, without notification, simply stop coming to the clinic after a few visits. The result is that efforts to measure change associated with clinic treatment in such cases must be based on psychiatric ratings rather than self-reports. Of the rating scales in current use, some are designed for measuring single dimensions such as depression2 Others, such as the Brief Psychiatric Rating Scale (BPRS)3 are designed for inpatients and thus do not cover the broad range of problems encountered in an outpatient clinic. The Global Assessment Scale (GAS)4 is designed to evaluate overall functioning on a single loo-point scale. One disadvantage of the GAS, however, is that the overall numerical rating does not identify the separate components that enter into life-adjustment and psychopathology. Another disadvantage is that short-term psychotherapy seldom produces much change in overall functioning. Thus, a need still exist for a rating instrument that
From the Albert Einstein College of Medicine/Monfefiore Medical Center, Bronx, NY. Address reprint requests to Robert Plutckik. Ph.D., Bronx Municipal Hospital Center, Pelkam Parkway S. and Eastchester Rd., Bronx, NY 10461. o 1990 by W. B. Saunders Company. 0010-440X/90/3102-0003$03.00/0 134
Comprehensive
Psychiatry,
Vol. 3 1, No. 2 (March/April),
1990: pp 134-l 39
MEASUREMENT
OF SYMPTOM
CHANGE
135
can be used quickly and reliably and that covers the full range of problems typical of psychiatric clinic outpatients. We describe the development of the Psychiatric Outpatient Rating Scale (PORS), a rating instrument that can be completed by a therapist and that has several important features: (a) It is designed with the primary objective of measuring symptoms and problems often found in psychiatric clinic outpatients; (b) it is quick and relatively simple to complete and score; (c) it provides a global score as well as individual scores for symptoms or problems; and (d) it covers the most frequently encountered problems which outpatients bring to a clinic. METHOD The PORS consists of 21 items (Table 1). A clinician rates the level of severity of each category ranging from 0 (not present) to 1 (minimal problem), to 2 (moderate problem), to 3 (severe problem), to 4 (very severe problem). These ratings may be cumulated to provide a total score. Consistent with reports of other patient rating scales,5.’ explicit definitions were not provided for the terms used, although measures of interjudge reliability were obtained. Interjudge reliability was determined in the following way. For 12 different patients, a supervising psychiatrist sat in with the psychiatric resident or psychology intern during the initial interview. At the end of the session, the interviewing clinician and the supervisor separately rated the patient for symptoms and problems on the PORS. Eight different residents and three different supervisors were involved. Product-moment correlations were then obtained between clinician and supervisor for each patient across the 21 items of the PORS. These correlations ranged from approximately + 50 to + .95 with a mean interjudge correlation of + .74 (significant at a level of more than .Ol). Thus, the PORS items apparently can be reliably judged by a clinician after an initial interview. This finding is consistent with those of other reports. For example, interrater reliability for individual items of the BPRS average +.75.* Similarly, interjudge reliabilities for the items of the Psychiatric Status Schedule’ range from a low of +.57 to a high of +.98 (mean correlation + .87). Interjudge reliability of the General Adult Inpatient Psychiatric Assessment Scale’ was also comparable. During an
Table 1. Frequency
Item 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 12. 13. 14. 15. 16. 17. 18. 19. 20. 21.
Self-esteem Depression Anxiety Dependency Social isolation Job or school problem lmpulsivity Sleep disturbance Marital conflicts Aggressive outbursts Problems with children Somatization Suicidal impulses or behaviors Sexual dysfunction Substance abuse Delusions Formal thought disorder Panic attacks Hallucinations Obsessive-compulsive rituals Phobias
NOTE. N = 86.
of Occurrence
of Symptoms
in Percent
No Problem
Minimal Problem
Moderate Problem
Severe Problem
3.5 5.8 11.5 19.8 20.9 33.7 40.7 43.0 50.9 55.8 59.3 59.3 60.5 66.3 70.9 79.1 81.4 83.7 83.7 86.0 86.0
9.3 31.4 20.9 15.1 15.1 15.1 16.3 26.6 7.0 10.5 11.6 23.3 29.1 16.3 11.6 5.8 7.0 4.7 4.7 7.0 2.3
37.2 40.7 43.0 27.9 27.9 26.7 25.6 18.6 15.1 17.4 9.3 7.0 8.1 9.3 8.1 9.3 8.1 5.8 8.1 3.5 7.0
50.0 22.1 24.5 37.2 36.0 24.4 17.4 12.8 27.9 16.3 19.8 10.5 2.3 8.1 9.3 5.9 3.5 5.9 3.5 3.5 4.6
PLUTCHIK ET AL
136
interviewer-observer setting, interjudge agreement (K) for individual items ranged from +.61 to 1.00 (mean + .80). Thus, clinicians apparently can rate individual descriptive items reliably on psychiatric assessment scales. The PGRS was developed by a committee of five experienced clinicians who have had extensive experience in psychiatric outpatient clinics. Their mean level of postgraduate experience was 12 years 4 months. The format and terminology of the scales underwent three revisions after several pilot studies and discussion with the therapists who actually used the scale. The setting for the study was a large municipal hospital outpatient clinic that handles approximately 10,000 visits and 1,200 new patients each year. The therapists are third-year psychiatric residents and psychology interns under supervision, as well as experienced social workers. In this clinic, two thirds of the patients leave within 12 sessions. Most receive individual therapy, although many patients also receive group, couple, and family therapy. For many of the patients, antianxiety or antidepression medication is prescribed. The present study used a retrospective design, based on patient records. Charts of 86 consecutive patients for whom at least a PORS initial evaluation had been completed were selected for inclusion. Of these 86 charts, 45 also contained a PORS discharge evaluation. The charts were read by an independent evaluator who was instructed to make two ratings on the GAS, one based on the symptomatology presented at intake and the other based on the description of the patient in the discharge summary. The GAS consists of a loo-point rating scale with descriptions of functional level provided at every lo-point increment; high scores indicate better levels of life functioning. To establish the interjudge reliability of the GAS rating, two evaluators independently rated 10 records each. The correlation between the two raters was +.82. The remaining records were then assessed by a single evaluator.
RESULTS
Table 2 shows general demographic and personal information about the sample of 86 patients studied. Men and women were almost equally represented. The age range was 18 to 67 years (mean, 39 years; SD = 12.7). The median number of outpatient sessions was 12 (range, 1 to 61 sessions). At the time of discharge, 26% of the patients were diagnosed as personality disorders, 23% as affective disorders, 19% as schizophrenic disorders, 19% as adjustment disorders, and 9% as anxiety disorder. Four percent of the sample had no psychiatric diagnosis at the time of discharge. Table 2. Demographic
and Diagnostic
Characteristics
Sex M =41 F = 45 Age (vr) Mean = 39.00 SD = 12.7 Range = 16-67 No. of sessions attended Median = 12 Mean = 12 SD = 14.2 Range = l-6 1 DSM-III discharge diagnoses (%) Personality disorder (26) Affective disorder (23) Schizophrenic disorder (19) Adjustment disorder (19) Anxiety disorder (9) No diagnosis given (4) NOTE. N = 86.
of the Sample
MEASUREMENT
OF SYMPTOM
CHANGE
137
Table 1 shows the frequency of occurrence of problems at the time of admission to the outpatient clinic, as rated by the clinician. Problems were evaluated as having different levels of severity; e.g., depression was a minimal problem for 31% of the sample, a moderate problem for 41% of the sample, and a severe problem for 22% of the sample. (To simplify Table 1, ratings for severe and very severe were combined.) The most common severe and moderate problems concerned self-esteem, social isolation, dependency, anxiety, and depression. The ubiquity of these symptoms suggests that for most outpatients they represent core complaints or accompany a wide range of presenting problems, or both. Symptoms suggestive of more serious pathology, such as might be typically found in an inpatient service, were not frequently encountered in this sample. The relatively few patients rated as having a thought disorder or hallucinations reflect a subgroup of patients referred to the outpatient clinic for follow-up treatment and medication after a period of inpatient hospitalization. Measurement
of Change
Table 3 summarizes the changes that occurred in the patients during their stay in the clinic. Of the original sample, 45 of the diagnosed patients had an evaluation made on each of the 21 items of the PORS at both intake and discharge. Mean scores for each item were obtained, and the initial mean rating for each item was compared with the discharge mean rating by a t test for correlated data. The results showed a significant decrease in severity of the problems for 7 of the 21 items. There was a significant decrease in severity of depression, anxiety, job or school problems or both, feelings of social isolation, temper outbursts, sleep Table 3. Significance
Tests for Differences
Item 1. 2. 3. 4. 5. 6. 7. a. 9. 10. 11. 12. 13. 14. 15. 16. 17. la. 19. 20. 21.
Delusions Sleep disturbance Anxien/ Job or school problem Aggressive outbursts Social isolation Depression Self esteem Substance abuse Panic attacks Problems with children Formal thought disorder Marital conflicts lmpulsivity Phobias Suicidal impulses or behavior Sexual dysfunction Obsessive compulsive rituals Somatization Hallucinations Dependency
NOTE. N = 45. lP< .Ol tP < .05
Between
Initial and Discharge
Ratings
Initial Mean
Discharge Mean
Initial SD
Discharge SD
t
0.49 1.04 2.00 I .5a 0.78 2.00 I .a9 2.51 0.31 0.40 1.04 0.44 1.75 0.97 0.47 0.49 0.73 0.31 0.80 0.31 2.07
0.16 0.58 1.53 1.24 0.53 1.64 1.62 2.24 0.38 0.31 0.82 0.29 1.62 0.82 0.38 0.29 0.55 0.22 0.69 0.24 2.02
0.99 1.19 I .28 1.39 1.22 1.37 0.83 0.99 0.63 1.10 1.52 0.92 1.67 1.12 1.10 0.76 1.16 0.70 1.16 0.82 1.42
0.52 0.97 1.08 1.23 1.01 1.30 0.89 1.03 0.75 0.91 1.32 0.79 1.54 1.16 0.91 0.87 0.87 0.64 1.02 0.88 1.32
2.91* 2.75’ 2.43T 2.4lT 2.41t 2.33T 2.01t 1.81 ’ 1.78 1.43 1.43 1.42 1.35 1.31 1.27 1.24 1.16 1.16 0.93 0.44 0.36
138
PLUTCHIK ET AL
problems, and delusions. All problems except substance abuse, which apparently increased during the course of therapy, showed a decrease in severity, although not of a statistically significant magnitude. The average number of therapy sessions for the 45 patients was 21 (SD, 14.9). Internal Reliability of the PORS
Until now, we have analyzed each item separately. Whether the items rated by the clinician form a psychometric scale is questionable, however. To determine this issue, the internal reliability of the group of items as a whole was computed by means of coefficient alpha. Internal reliability for the initial sample of 86 patients was .70. For the subgroup of 45 patients who were evaluated initially and at the time of discharge, the coefficient alpha was .84. Thus, the items of the PORS rating scales appear to be fairly highly intercorrelated and the PORS apparently can be treated as a scale whose total is a meaningful reflection of the individual item responses. Concurrent Validity
A further analysis was performed to determine the relationship between a global assessment of the level of dysfunction of the patient and the total score on the PORS. The global assessment was made by the GAS. The GAS score was determined for 35 patients and then correlated against the PORS scores of the same patients. The correlation was -.64, significant at the .OOl level. (The minus sign reflects the fact that high scores on the GAS express good functioning whereas high scores on the PORS reflect poor functioning.) Construct Validity
The final analysis was the determination of the correlation between total PORS change scores and number of sessions. This was -.37 (P < .02), indicating that patients who remained in treatment tended to have greater decreases in problem severity. The same analysis was computed between the number of sessions and GAS change scores. The correlation +.40 (P -z .Ol) indicating again that the longer patients remained in treatment, the greater the increase in their overall functioning. DISCUSSION
We present data obtained on a group of outpatients using a new 21-item symptom rating scale, the PORS. The internal reliability of the PORS was high, its interjudge reliability was high, and its correlation with the well-known GAS was -.64, indicating a reasonable degree of concurrent validity. As a measure of construct validity, the PORS correlated .37 with number of sessions of psychotherapy, indicating that patients who stayed in treatment longer tended to improve more. This finding is consistent with the conclusion of Howard et al.,9 who reported a dose-effect relationship based on a metaanalysis of 15 clinical studies of psychotherapy. The percentage of improved patients increased rapidly up to about 26 sessions and then gradually leveled off. A roughly similar observation was made in the present study, although the treatments were quite diverse and included individual and group therapy as well as psychotropic medication. The population of this outpatient clinic is quite heterogeneous. For the most part, the patients are of a lower middle class social status and consider the clinic a place
MEASUREMENT
OF SYMPTOM
CHANGE
139
where they can try to solve problems in a relatively brief period. A certain proportion of the patients are former psychiatric inpatients who have been referred to the outpatient clinic for follow-up treatment. Such treatment consists in most cases of psychotropic medication and supportive individual or group sessions. Because of the heterogeneity of the population, some of the symptoms that improved significantly are worth considering. These are (in order of magnitude of change) delusions, sleep disturbance, anxiety, aggressive outbursts, and depression. Most of these problems tend to benefit the most from psychotropic medication and, considering the relatively short-term therapy received by these patients, the average changes observed may reflect the benefits of drugs as much as the benefits of psychotherapy. In contrast, there was relatively little change in such symptoms as dependency, obsessive compulsive behavior, impulsivity, and marital conflicts. This observation may reflect the fact that these symptoms represent personality traits or chronic tendencies, characteristics that are less affected by drugs and are influenced only by much therapeutic interaction. A similar point was made by Weissman et al.” We cannot specify exactly what proportion of improvement results from drugs and what proportion results from psychotherapy, although Conte et al.” have shown in a metaanalysis that some summation of drug plus psychotherapy effects does occur. The current findings are consistent with the possibility of a drug/ psychotherapy interaction. The PORS may be a useful tool for studying change in psychotherapy. It is simple and easy to use and covers a wide range of outpatient problems and symptoms. It may also serve to sensitize the clinician to consider issues that might otherwise be missed, thereby enhancing therapeutic effectiveness. REFERENCES 1. Plutchik R, Conte HR: Quantitative assessment of personality disorders, in Michels R, Cavenar JO Jr, Brodie HKH, et al (eds) Psychiatry, vol. 1, Philadelphia, Lippincott, 1985 2. Hamilton M: A rating scale for depression. J Nemo1 Neurosurg Psychiatry 2356-62, 1960 3. Overall JE, Gorham DR: The Brief Psychiatric Rating Scale. Psycho1 Rep 10:799-812, 1962 4. Endicott J, Spitzer RL, Fleiss JL, et al: The Global Assessment Scale. Arch Gen Psychiatry 331766771, 1976 5. Spitzer RL, Endicott J, Fleiss JL, et al: The Psychiatric Status Schedule: A technique for evaluating psychopathology and impairment in role functioning. Arch Gen Psychiatry 23:41-S& 1970 6. Overall JE, Hollister LE, Pichot P: Major psychiatric disorders: A four-dimensional model. Arch Gen Psychiatry 15:146-151, 1967 7. Summers WK, Marsh GM, Chiong B, et al: The General Adult Inpatient Psychiatric Assessment Scale (GAIPAS). Psychiatry Res 10:217-236, 1983 8. Hedlund JL, Vieweg BW: The Brief Psychiatric Rating Scale (BPRS): A comprehensive review. J Operational Psychiatry 11:48-65, 1980 9. Howard KI, Kopta SM, Krause MS, et al: The dose-effect relationship in psychotherapy. Am Psycho1 41:159-164,1986 10. Weissman MM, Klerman GL, Prusoff GA, et al: Depressed outpatients: One year after treatment with drugs and/or interpersonal psychotherapy. Arch Gen Psychiatry 38:51-55, 1981 11. Conte HR, Plutchik R, Wild KV, et al: Combined psychotherapy and pharmacotherapy for depression. Arch Gen Psychiatry 43:471-479, 1986
