Pwcepturrl and Motor Skills, 1978,47,851-856.
@ Perceptual and Motor Skills 1978
ON-LINE MEASUREMENT OF APHASIC SPEECH' ANN PACKMAN AND ROGER J. INGHAM Cumberland CoUege of Health Sciences Summary.Seven response categories, devised for the concurrent evaluation of aphasic speech, were investigated for their reliability. The spontaneous speech of 5 aphasic subjects was rated on-line by 4 clinicians using 7 response categories. Agreement among the 4 judges, for both inter-judge and intra-judge reliability, varied for each subjea and was not high enough to support the concurrent use of the seven response categories for the on-line investigation of aphasic speech in all subjects. Ways in which the reliability of the categories might be increased are discussed.
Only limited use has been made of operant methodology in the modification of aphasic speech. Probably one of the major reasons for this has been the difficulty of identifying reliable target behaviors (Packman & Ingham, 1977). A promising study by Goodkin and associates (Goodkin, Diller, & Shah, 1973) appears to be the only reported contingency-management treatment program in which the reliability of response classes has been identified. The verbal interaction of 23 pairs of aphasic subjects and their spouses was investigated in the study by Goodkin, et aE. The aim was to determine whether a spouse could be trained to alter the speaking behavior of an aphasic by systematically varying verbal contingencies. Seven response categories of verbal behavior were used to describe aphasic speech and eight response categories were used to describe the speech of their spouses. High reliability was found across the 15 categories, for two aphasic subjects and their spouses, over seven occasions (Goodkin, 1968). The author concluded:
. . . the above cases suggest that these categories can be scored with sufficient reliability to be useful to the aphasia researcher and to the speech therapist in evaluating response patterns and changes. Concurrent response categories such as these might well be useful in objectively evaluating speech and in validating subjective impressions of speech (1968, pp. 1039-1040). A procedure for "on-line" measurement of spontaneous aphasic speech would have numerous applications in experimental and clinical research, especially where response-contingent schedules are employed. Such a measurement procedure must be "on-line" if stimuli are to be presented contingent on, and immediately following, the responses. The seven aphasic response categories devised by Goodkin would provide a suitable measurement procedure and could be applied in clinical settings where a spouse is not present. Preliminary 'This study was supported by a grant from the National Health and Medical Research Council of Australia (Grant No. 74/345). The authors wish to thank Ken Wade for his assistance in data analysis, and Lyn ~ e v i n ~ t o Sue n , Jackson, Jenine Klamus, and Ola Todd for providing subjects. Requests for reprints should be sent to Dr. R. J. Ingham, School of Communication Disorders, Cumberland College of Health Sciences, P. 0. Box 170, Lidcombe, N.S.W., Australia 2141.
852
A. PACKMAN & R. J. INGHAM
investigations into the clinical application of similar procedures, where spouses were not involved, ate reported by Goodkin ( 1966, 1969). For a number of reasons, however, further investigation is required before Goodkin's aphasic response categories can be generally adopted for this purpose. At present the application of the response categories is restricted by the fact that the reliability measures in Goodkids study were computed using the responses of the spouses as well as the aphasics. Consequently, an investigation is needed to establish the reliability of the seven aphasic response categories. In addition, in view of the large range of individual differences found in aphasics, it seems unwise to assume that the high reliability found by Goodkin applies to all aphasic subjects. To investigate this, it would be necessary to increase the number of subjects and to obtain reliability measures for each category in each subject. The aim of the present study was to follow the procedure used by Goodkin (1968) without including spouses. The number of subjects was increased to five and the number of judges was increased from two to f m .
The subjects were five adults, three males and two females, whose ages ranged from 47 to 70 yr. They had been diagnosed as aphasic by a speech clinician, using either the Minnesota Test for the Differential Diagnosis of Aphasia (Schuell, 1965) or the Porch Index of Communicative Ability (Porch, 1967). All subjects had suffered a cerebro-vascular accident to the left hemisphere and all were currently receiving therapy.
A p paratas All recordings were made with a Revox A77 tape recorder and AKG DI90C microphone. The same tape recorder was used for the presentation of recordings via Sony DR7 headphones to the judges. Pro cedare The procedure followed that used by Goodkin (1968). The 10 stimulus questions were altered slightly to suit local conditions (see Appendix 1 ) . The experimenter asked each of the five subjects the 10 questions allowing 2 min. for each reply. The experimenter repeated a question when necessary and encouraged the subject to continue talking during the 2 min. but without cueing or prompting. The recordings of these five sessions were played independently to four experienced clinicians on two occasions, separated by a period of 2 wk. The procedures used for data collection and for training judges are not described in detail in Goodkin's study. In the present study on each occasion a tally sheet with the definitions of the seven aphasic response categories (see Appendix 2 ) was given to each judge with instructions to mark the tally sheet every time she
ON-LINE MEASUREMENT OF APHASIC SPEECH
853
judged that a response was emitted within any of the categories. The tally sheets were designed to display the frequency of responding within each category, for each 2-min. speech segment. Each judge was allowed 30 min. for discussion with the experimenter about the categories, as described by Goodkin. The judges were instructed not to replay any portion of the tape.
RESULTS The data from this study were analyzed in several ways. Table 1 shows the total number of category responses rated by each of the four judges for each subject. At this gross level of analysis, it is evident that there is considerable variation in the judges' scores for each subject. TABLB 1 Judge 1
Subject 3
2
4
5
Kendall's Coefficient of Concordance, W (Siegel, 1956), was used to measure the agreement between category scores from all four judges. Values of W were obtained for each subject's session and for each category within each subject's session. To calculate the sessional W, the totals for each category were . used. However, in order to calculate the category W, it was necessary to use the totals for each of the 10 2-min. speech segments which made up each category total. This procedure therefore encompasses more stringent measures of category-response frequency over time. These results are shown in Table 2. There TABLB 2 KENDALLW FOREACH SUBJECT'S SESSION ( k = 4. N = 7 ) AND FOR EACH CATEGORY IN EACH SESSION ( k = 4, N = 10) Inter-judge agreement W Subject Session Category 1 2
3 4 5
6 4:p< .05; f p
@ Perceptual and Motor Skills 1978
ON-LINE MEASUREMENT OF APHASIC SPEECH' ANN PACKMAN AND ROGER J. INGHAM Cumberland CoUege of Health Sciences Summary.Seven response categories, devised for the concurrent evaluation of aphasic speech, were investigated for their reliability. The spontaneous speech of 5 aphasic subjects was rated on-line by 4 clinicians using 7 response categories. Agreement among the 4 judges, for both inter-judge and intra-judge reliability, varied for each subjea and was not high enough to support the concurrent use of the seven response categories for the on-line investigation of aphasic speech in all subjects. Ways in which the reliability of the categories might be increased are discussed.
Only limited use has been made of operant methodology in the modification of aphasic speech. Probably one of the major reasons for this has been the difficulty of identifying reliable target behaviors (Packman & Ingham, 1977). A promising study by Goodkin and associates (Goodkin, Diller, & Shah, 1973) appears to be the only reported contingency-management treatment program in which the reliability of response classes has been identified. The verbal interaction of 23 pairs of aphasic subjects and their spouses was investigated in the study by Goodkin, et aE. The aim was to determine whether a spouse could be trained to alter the speaking behavior of an aphasic by systematically varying verbal contingencies. Seven response categories of verbal behavior were used to describe aphasic speech and eight response categories were used to describe the speech of their spouses. High reliability was found across the 15 categories, for two aphasic subjects and their spouses, over seven occasions (Goodkin, 1968). The author concluded:
. . . the above cases suggest that these categories can be scored with sufficient reliability to be useful to the aphasia researcher and to the speech therapist in evaluating response patterns and changes. Concurrent response categories such as these might well be useful in objectively evaluating speech and in validating subjective impressions of speech (1968, pp. 1039-1040). A procedure for "on-line" measurement of spontaneous aphasic speech would have numerous applications in experimental and clinical research, especially where response-contingent schedules are employed. Such a measurement procedure must be "on-line" if stimuli are to be presented contingent on, and immediately following, the responses. The seven aphasic response categories devised by Goodkin would provide a suitable measurement procedure and could be applied in clinical settings where a spouse is not present. Preliminary 'This study was supported by a grant from the National Health and Medical Research Council of Australia (Grant No. 74/345). The authors wish to thank Ken Wade for his assistance in data analysis, and Lyn ~ e v i n ~ t o Sue n , Jackson, Jenine Klamus, and Ola Todd for providing subjects. Requests for reprints should be sent to Dr. R. J. Ingham, School of Communication Disorders, Cumberland College of Health Sciences, P. 0. Box 170, Lidcombe, N.S.W., Australia 2141.
852
A. PACKMAN & R. J. INGHAM
investigations into the clinical application of similar procedures, where spouses were not involved, ate reported by Goodkin ( 1966, 1969). For a number of reasons, however, further investigation is required before Goodkin's aphasic response categories can be generally adopted for this purpose. At present the application of the response categories is restricted by the fact that the reliability measures in Goodkids study were computed using the responses of the spouses as well as the aphasics. Consequently, an investigation is needed to establish the reliability of the seven aphasic response categories. In addition, in view of the large range of individual differences found in aphasics, it seems unwise to assume that the high reliability found by Goodkin applies to all aphasic subjects. To investigate this, it would be necessary to increase the number of subjects and to obtain reliability measures for each category in each subject. The aim of the present study was to follow the procedure used by Goodkin (1968) without including spouses. The number of subjects was increased to five and the number of judges was increased from two to f m .
The subjects were five adults, three males and two females, whose ages ranged from 47 to 70 yr. They had been diagnosed as aphasic by a speech clinician, using either the Minnesota Test for the Differential Diagnosis of Aphasia (Schuell, 1965) or the Porch Index of Communicative Ability (Porch, 1967). All subjects had suffered a cerebro-vascular accident to the left hemisphere and all were currently receiving therapy.
A p paratas All recordings were made with a Revox A77 tape recorder and AKG DI90C microphone. The same tape recorder was used for the presentation of recordings via Sony DR7 headphones to the judges. Pro cedare The procedure followed that used by Goodkin (1968). The 10 stimulus questions were altered slightly to suit local conditions (see Appendix 1 ) . The experimenter asked each of the five subjects the 10 questions allowing 2 min. for each reply. The experimenter repeated a question when necessary and encouraged the subject to continue talking during the 2 min. but without cueing or prompting. The recordings of these five sessions were played independently to four experienced clinicians on two occasions, separated by a period of 2 wk. The procedures used for data collection and for training judges are not described in detail in Goodkin's study. In the present study on each occasion a tally sheet with the definitions of the seven aphasic response categories (see Appendix 2 ) was given to each judge with instructions to mark the tally sheet every time she
ON-LINE MEASUREMENT OF APHASIC SPEECH
853
judged that a response was emitted within any of the categories. The tally sheets were designed to display the frequency of responding within each category, for each 2-min. speech segment. Each judge was allowed 30 min. for discussion with the experimenter about the categories, as described by Goodkin. The judges were instructed not to replay any portion of the tape.
RESULTS The data from this study were analyzed in several ways. Table 1 shows the total number of category responses rated by each of the four judges for each subject. At this gross level of analysis, it is evident that there is considerable variation in the judges' scores for each subject. TABLB 1 Judge 1
Subject 3
2
4
5
Kendall's Coefficient of Concordance, W (Siegel, 1956), was used to measure the agreement between category scores from all four judges. Values of W were obtained for each subject's session and for each category within each subject's session. To calculate the sessional W, the totals for each category were . used. However, in order to calculate the category W, it was necessary to use the totals for each of the 10 2-min. speech segments which made up each category total. This procedure therefore encompasses more stringent measures of category-response frequency over time. These results are shown in Table 2. There TABLB 2 KENDALLW FOREACH SUBJECT'S SESSION ( k = 4. N = 7 ) AND FOR EACH CATEGORY IN EACH SESSION ( k = 4, N = 10) Inter-judge agreement W Subject Session Category 1 2
3 4 5
6 4:p< .05; f p
