Journal oj Consulting and Clinical Psychology 1975, Vol. 43, No. 6, 810-815
Effects of Prognostic Information on Global Ratings of Psychotherapy Outcome Grady L. Blackwood, Jr., Hans H. Strupp, and Laurence A. Bradley Vanderbilt University Raters viewed a videotape recording of an interview with a patient after having read one of three types of pretherapy information about the patient. One group of raters was presented information associated with a good prognosis, one group was presented information associated with a poor prognosis, and the last group received neutral information about the patient. One half of the members of each group were told that the interview was made at the termination of therapy, the other half were told that it was a 1-year follow-up interview. All raters then completed scales regarding patient's level of adjustment, psychic distress, amount of change, and success of patient's therapy. Only ratings of pretherapy to posttherapy patient change varied as a function of prognostic information. The results suggest that although poor prognosis patients are perceived as exhibiting more change than good prognosis patients, only final level of functioning determines global judgments of psychotherapy outcome.
The evaluation of the outcome of psychotherapeutic treatment continues to be one of the most active and challenging areas of research in psychotherapy (Bergin & Suinn, 1975; Meltzoff & Kornreich, 1970). Among many diverse criterion measures employed to evaluate outcome (Bergin, 1971), two broad categories can be distinguished: global assessments of the success of therapy and psychometric measures of change. Although several recent reviews suggest that emphasis should be placed on further development of specific measures of patient change (Bergin, 1971; Kiesler, 1971; Strupp & Bergin, 1969), global assessment procedures remain the more popular means of outcome evaluation (Luborsky, Chandler, Auerbach, Cohen, & Bachrach, 1971). Given the popularity of global ratings of outcome, it is important that variables that affect therapists' ratings be examined. Mintz (1972) has suggested that a major factor affecting global ratings may be the therapist's expectation of the degree to which a patient might benefit from psychotherapy, that is, prognosis. Isolating three prominent variables that significantly affect prognostic ratings (chronicity and severity of problems and social
assets), Mintz examined the effects of perceived prognosis on global ratings of the success of therapy. When raters were presented with hypothetical patients who had identical pretherapy and posttherapy scores on a mock personality inventory but who differed in prognosis (good vs. poor), they rated therapy as more successful for those patients expected to have greater success than for those with a poor prognosis. Further, in another study, Mintz (1972) found that judges weight the amount of change differently at different levels of functioning, viewing a small improvement in adjustment at an adequate level of functioning as more valuable than a larger improvement at a less adequate level of functioning. These studies led Mintz (1972) to conclude that "the better adjusted the patient at the end of therapy, the better the outcome ratings regardless of the amount of change involved" (p. 18). These findings suggest that although good and poor prognosis patients may end therapy at the same level of adjustment according to objective data, subjectively, the meaning of numerical endpoints on test data differs as a function of prognosis. Evidently, raters in the Mintz studies felt, perhaps on the basis of experience, that a high endpoint (in terms of Requests for reprints should be sent to Grady L. level of adjustment) is more valid for a good Blackwood, Jr., Department of Psychology, 134 Wesley Hall, Vanderbilt University, Nashville, Tennessee prognosis patient than for a poor prognosis 37240. patient. Indeed, Mintz reported that when con810
811
PROGNOSTIC INFORMATION fronted with the data, raters argued that even when both good and poor prognosis patients end therapy with equally good posttherapy adjustment ratings, only the good prognosis patients can be expected to "consolidate and maintain" gains made in therapy. In light of these findings, it appears that global ratings of outcome place the poor prognosis patient at a disadvantage. First, although he must change more than a good prognosis patient to arrive at the same level of posttherapy adjustment, amount of change is not reflected in the success rating of his therapy. Second, when he does terminate therapy at an adjustment level as high as that arrived at by the good prognosis patient, the validity of his achievement is suspect. Thus, there appears, to be a tendency for global measures of therapy outcome to favor the good prognosis patient over the poor prognosis patient. If Mintz's (1972) results are valid and reliable, the meaning and use of global outcome measures in psychotherapy research should be carefully examined. However, the studies reported by Mintz contained several methodological deficiencies that render suspect the internal validity of the findings (Campbell & Stanley, 1971). These were as follows: (a) a small sample of raters (.V < 10); (b) heterogeneity of raters (medical doctors, clinical psychologists, research assistants); and (c) lack of a control group. The present study examined the effects of differential prognostic information on global ratings of outcome under more appropriate experimental conditions than those originally employed by Mintz. The study included a "neutral" prognostic information condition as a control treatment as well as a large number of raters with similar academic and clinical experience. In addition, the study examined global ratings of therapy outcome as a function of the time of assessment (at termination of therapy or 1 year -following termination) in order to determine whether evidence indicating the patient had maintained his therapeutic gains following termination of therapy would affect any possible relationships between differential prognosis and global assessments of outcome.
METHOD Subjects Thirty graduate students in clinical psychology from Vanderbilt University and George Peabody College for Teachers participated as raters. All students had at least 1 year of experience in individual psychotherapy and were currently seeing individual patients.
Material Videotape. A staff member of the Vanderbilt-Peabody-Scarritt Interuniversity Psychological and Counseling Center and a Vanderbilt drama student were hired to portray a therapist and patient. A videotape recording of an interview between the therapist and the patient was made with a Model AV3600 Sony videotape recording unit. During the 15-minute interview the patient described himself as better off than he was before therapy and more actively engaged in work and social activities. The therapist remained silent throughout most of the tape, only occasionally asking for further information or clarification of a statement. Rating booklets. Standard instructions and information about the study and dependent measures were presented in two rating booklets, an "information and instruction booklet" used before raters saw the videotape and a "rating booklet" used after the tape. Each information and instruction booklet contained one of three types of intake (pretherapy) information about the patient (good, poor, or neutral) presented in typed form on the standard intake interview summary sheet used by the center and a note indicating whether the videotape about to be seen was a segment of a termination or a follow-up interview.
Dependent Measures Dependent measures consisted of eight Likert-type scales presented in the two booklets. The four scales presented in the information booklet were concerned with the patient's prognosis, level of adjustment, psychic distress, and severity of problems at the beginning of therapy. The rating booklet contained similar rating scales on the success of therapy, the patient's level of adjustment and psychic distress at the time of the interview, and the amount of change shown by the patient. Raters made their ratings along the 7-point scales with 1 indicating an extremely negative aspect, 4 an average aspect, and 7 an extremely positive aspect of the variable (except in the case of psychic distress in which the endpoints were reversed).
Procedure The design was a 3 X 2 factorial with the betweensubjects variables prognostic information (good, poor, or neutral) and time (termination or follow-up) and the dependent variable, ratings (Scales 1-8). Raters were randomly assigned to one of the three prognostic information conditions. One half of the raters in each condition were randomly selected to receive the information booklet information that the videotaped interview took place at the termination of therapy, and the
G. L. BLACKWOOD, JR., H. H. STRUPP, AND L. A. BRADLEY
812
other half were informed that the interview was a 1-year follow-up. Each group of raters was met by the experimenter, who explained that they were about to view a segment of an interview of a patient who was seen at the counseling center and that they would be asked to make some ratings on the basis of their assessment of the patient after viewing the videotape. Raters were then presented with the information booklet, which provided all further instructions and prognostic information regarding the patient. The raters in the neutral information condition received information such as the patient's age, place of birth, hobbies, year in school, number of siblings, and age of parents. The good prognosis raters received the same information and, in addition, were informed that the patient came to the counseling center mildly depressed due to a very recent breakup with a girlfriend whom he had been dating for 3 years. The patient was further described as a socially active, "A" student from an upper-middle-class family. The poor prognosis raters received the same information as the neutral group and, in addition, were informed that the patient came to the counseling center severely depressed over the death of a girlfriend that occurred 3 years ago. Further, it was reported that since her death he had suffered recurrent episodes of extreme depression, was socially isolated, a "C —" student, and that he came from a lower-class background. After reading this information, all raters completed the four rating scales included in the information booklet and then viewed the same videotape recording of the patient's interview. At the conclusion of the videotape, raters were administered the rating booklet and made ratings on its four scales. When all raters had finished, the group was debriefed.
RESULTS A series of 3 X 2 analyses of variance were performed with the between-subjects variables, prognosis information (good, neutral, or poor) and time (termination or follow-up), and the dependent variable, ratings (Scales 1-8). Information and Instruction Booklet Ratings As can be seen in Table 1, significant main effects of prognosis information were found on ratings of the patient's prognosis, F(2, 24) = 12.10, p < .001, £>2 = .45, pretherapy level of adjustment, F(2, 24) = 48.88, p < .001, u>2 = .78, pretherapy psychic distress, F(2, 24) = 22.14, p < .001, A2 = .59, and severity of the patient's problems, F(2, 24) = 17.58, p < .001, w2 = .53. These analyses were performed in order to determine whether raters' perceptions of the patient were consistent with the intake interview information provided them; thus, a priori t tests between the means were performed. It was found that raters in the good information condition gave the patient a significantly better prognosis than did raters in the poor information condition,
Effects of Prognostic Information on Global Ratings of Psychotherapy Outcome Grady L. Blackwood, Jr., Hans H. Strupp, and Laurence A. Bradley Vanderbilt University Raters viewed a videotape recording of an interview with a patient after having read one of three types of pretherapy information about the patient. One group of raters was presented information associated with a good prognosis, one group was presented information associated with a poor prognosis, and the last group received neutral information about the patient. One half of the members of each group were told that the interview was made at the termination of therapy, the other half were told that it was a 1-year follow-up interview. All raters then completed scales regarding patient's level of adjustment, psychic distress, amount of change, and success of patient's therapy. Only ratings of pretherapy to posttherapy patient change varied as a function of prognostic information. The results suggest that although poor prognosis patients are perceived as exhibiting more change than good prognosis patients, only final level of functioning determines global judgments of psychotherapy outcome.
The evaluation of the outcome of psychotherapeutic treatment continues to be one of the most active and challenging areas of research in psychotherapy (Bergin & Suinn, 1975; Meltzoff & Kornreich, 1970). Among many diverse criterion measures employed to evaluate outcome (Bergin, 1971), two broad categories can be distinguished: global assessments of the success of therapy and psychometric measures of change. Although several recent reviews suggest that emphasis should be placed on further development of specific measures of patient change (Bergin, 1971; Kiesler, 1971; Strupp & Bergin, 1969), global assessment procedures remain the more popular means of outcome evaluation (Luborsky, Chandler, Auerbach, Cohen, & Bachrach, 1971). Given the popularity of global ratings of outcome, it is important that variables that affect therapists' ratings be examined. Mintz (1972) has suggested that a major factor affecting global ratings may be the therapist's expectation of the degree to which a patient might benefit from psychotherapy, that is, prognosis. Isolating three prominent variables that significantly affect prognostic ratings (chronicity and severity of problems and social
assets), Mintz examined the effects of perceived prognosis on global ratings of the success of therapy. When raters were presented with hypothetical patients who had identical pretherapy and posttherapy scores on a mock personality inventory but who differed in prognosis (good vs. poor), they rated therapy as more successful for those patients expected to have greater success than for those with a poor prognosis. Further, in another study, Mintz (1972) found that judges weight the amount of change differently at different levels of functioning, viewing a small improvement in adjustment at an adequate level of functioning as more valuable than a larger improvement at a less adequate level of functioning. These studies led Mintz (1972) to conclude that "the better adjusted the patient at the end of therapy, the better the outcome ratings regardless of the amount of change involved" (p. 18). These findings suggest that although good and poor prognosis patients may end therapy at the same level of adjustment according to objective data, subjectively, the meaning of numerical endpoints on test data differs as a function of prognosis. Evidently, raters in the Mintz studies felt, perhaps on the basis of experience, that a high endpoint (in terms of Requests for reprints should be sent to Grady L. level of adjustment) is more valid for a good Blackwood, Jr., Department of Psychology, 134 Wesley Hall, Vanderbilt University, Nashville, Tennessee prognosis patient than for a poor prognosis 37240. patient. Indeed, Mintz reported that when con810
811
PROGNOSTIC INFORMATION fronted with the data, raters argued that even when both good and poor prognosis patients end therapy with equally good posttherapy adjustment ratings, only the good prognosis patients can be expected to "consolidate and maintain" gains made in therapy. In light of these findings, it appears that global ratings of outcome place the poor prognosis patient at a disadvantage. First, although he must change more than a good prognosis patient to arrive at the same level of posttherapy adjustment, amount of change is not reflected in the success rating of his therapy. Second, when he does terminate therapy at an adjustment level as high as that arrived at by the good prognosis patient, the validity of his achievement is suspect. Thus, there appears, to be a tendency for global measures of therapy outcome to favor the good prognosis patient over the poor prognosis patient. If Mintz's (1972) results are valid and reliable, the meaning and use of global outcome measures in psychotherapy research should be carefully examined. However, the studies reported by Mintz contained several methodological deficiencies that render suspect the internal validity of the findings (Campbell & Stanley, 1971). These were as follows: (a) a small sample of raters (.V < 10); (b) heterogeneity of raters (medical doctors, clinical psychologists, research assistants); and (c) lack of a control group. The present study examined the effects of differential prognostic information on global ratings of outcome under more appropriate experimental conditions than those originally employed by Mintz. The study included a "neutral" prognostic information condition as a control treatment as well as a large number of raters with similar academic and clinical experience. In addition, the study examined global ratings of therapy outcome as a function of the time of assessment (at termination of therapy or 1 year -following termination) in order to determine whether evidence indicating the patient had maintained his therapeutic gains following termination of therapy would affect any possible relationships between differential prognosis and global assessments of outcome.
METHOD Subjects Thirty graduate students in clinical psychology from Vanderbilt University and George Peabody College for Teachers participated as raters. All students had at least 1 year of experience in individual psychotherapy and were currently seeing individual patients.
Material Videotape. A staff member of the Vanderbilt-Peabody-Scarritt Interuniversity Psychological and Counseling Center and a Vanderbilt drama student were hired to portray a therapist and patient. A videotape recording of an interview between the therapist and the patient was made with a Model AV3600 Sony videotape recording unit. During the 15-minute interview the patient described himself as better off than he was before therapy and more actively engaged in work and social activities. The therapist remained silent throughout most of the tape, only occasionally asking for further information or clarification of a statement. Rating booklets. Standard instructions and information about the study and dependent measures were presented in two rating booklets, an "information and instruction booklet" used before raters saw the videotape and a "rating booklet" used after the tape. Each information and instruction booklet contained one of three types of intake (pretherapy) information about the patient (good, poor, or neutral) presented in typed form on the standard intake interview summary sheet used by the center and a note indicating whether the videotape about to be seen was a segment of a termination or a follow-up interview.
Dependent Measures Dependent measures consisted of eight Likert-type scales presented in the two booklets. The four scales presented in the information booklet were concerned with the patient's prognosis, level of adjustment, psychic distress, and severity of problems at the beginning of therapy. The rating booklet contained similar rating scales on the success of therapy, the patient's level of adjustment and psychic distress at the time of the interview, and the amount of change shown by the patient. Raters made their ratings along the 7-point scales with 1 indicating an extremely negative aspect, 4 an average aspect, and 7 an extremely positive aspect of the variable (except in the case of psychic distress in which the endpoints were reversed).
Procedure The design was a 3 X 2 factorial with the betweensubjects variables prognostic information (good, poor, or neutral) and time (termination or follow-up) and the dependent variable, ratings (Scales 1-8). Raters were randomly assigned to one of the three prognostic information conditions. One half of the raters in each condition were randomly selected to receive the information booklet information that the videotaped interview took place at the termination of therapy, and the
G. L. BLACKWOOD, JR., H. H. STRUPP, AND L. A. BRADLEY
812
other half were informed that the interview was a 1-year follow-up. Each group of raters was met by the experimenter, who explained that they were about to view a segment of an interview of a patient who was seen at the counseling center and that they would be asked to make some ratings on the basis of their assessment of the patient after viewing the videotape. Raters were then presented with the information booklet, which provided all further instructions and prognostic information regarding the patient. The raters in the neutral information condition received information such as the patient's age, place of birth, hobbies, year in school, number of siblings, and age of parents. The good prognosis raters received the same information and, in addition, were informed that the patient came to the counseling center mildly depressed due to a very recent breakup with a girlfriend whom he had been dating for 3 years. The patient was further described as a socially active, "A" student from an upper-middle-class family. The poor prognosis raters received the same information as the neutral group and, in addition, were informed that the patient came to the counseling center severely depressed over the death of a girlfriend that occurred 3 years ago. Further, it was reported that since her death he had suffered recurrent episodes of extreme depression, was socially isolated, a "C —" student, and that he came from a lower-class background. After reading this information, all raters completed the four rating scales included in the information booklet and then viewed the same videotape recording of the patient's interview. At the conclusion of the videotape, raters were administered the rating booklet and made ratings on its four scales. When all raters had finished, the group was debriefed.
RESULTS A series of 3 X 2 analyses of variance were performed with the between-subjects variables, prognosis information (good, neutral, or poor) and time (termination or follow-up), and the dependent variable, ratings (Scales 1-8). Information and Instruction Booklet Ratings As can be seen in Table 1, significant main effects of prognosis information were found on ratings of the patient's prognosis, F(2, 24) = 12.10, p < .001, £>2 = .45, pretherapy level of adjustment, F(2, 24) = 48.88, p < .001, u>2 = .78, pretherapy psychic distress, F(2, 24) = 22.14, p < .001, A2 = .59, and severity of the patient's problems, F(2, 24) = 17.58, p < .001, w2 = .53. These analyses were performed in order to determine whether raters' perceptions of the patient were consistent with the intake interview information provided them; thus, a priori t tests between the means were performed. It was found that raters in the good information condition gave the patient a significantly better prognosis than did raters in the poor information condition,
