Letters to the Editor A Questionable Consistency: Response to Fitch

confounded in this study with reliability of the instrument itself, which should be assessed by repeatedly submitting the same stored acoustic signal to the Visi-Pitch device for analysis. Failure to perform such tests might be taken as an implicit assumption of the intrinsic reliability of the instrumentation on Fitch's part. As printed, the only measure of consistency provided in this article is a statistical comparison (i.e., F-ratio) of means, standard deviations, and ranges for two trials for each gender group. This type of group data alone are of questionable relevance when assessing test stability (Horst, 1966). Despite the fact that ANOVA and correlation procedures share the same underlying mathematical model (i.e., multivariate general linear hypothesis), correlational data provide information concerningg the degree of relationship which the F-ratio does not convey" (Guilford, 1965; p. 314). It is quite possible to achieve the same group means, standard deviations, and ranges on two task administrations while having individual subjects perform very differently (i.e., unreliably) across these occasions. As a case in point, consider the data set in Table 1. This hypothetical data set was constructed to approximate the reading task values for mean fundamental frequency reported for males in Fitch's study. Test 1 represents an initial administration. Retests A, B, and C represent three alternative distributions yielding identical summary statistics on a second administration. For Retest A there is a high degree of similarity of subject performance in comparison to Test 1 (r = .986, p < .01). For Retest B, there is an inverse relationship in comparison to Test 1 (r = -. 900, p < .01). In Retest C the relationship to Test 1 is random (r = .083, p > .50). Obviously, only Retest A can be regarded as a consistent pattern of subject performance relative to Test 1. The same point can be made for all of the comparisons reported by Fitch. It is this "stability of the rank orders of individuals over a period of time" that constitutes the essence of test-retest methodology (Guilford, 1965; p. 451).

(1990) With increasing frequency in recent years, a variety of automated and semiautomated quantitative speech performance measures, derived from digital electronics, are finding their way into the routine practice of speech-language pathology. Although quantification is clearly a desirable clinical goal, we should not be blinded by apparent or superficial precision of such gadgetry or fail to question fundamental assumptions (i.e., are these valid and reliable measures of the behavior of interest?). A recent study by Fitch (1990) sought to examine several such measures of voice fundamental frequency across repeated productions of various speaking tasks using an IBM implementation of the Kay Elemetrics Visi-Pitch device. As the author indicates, there is a pressing need for studies of this type due to the widespread clinical acceptance of Visi-Pitch and its common utilization in the assessment and treatment of voice-disordered patients. Fitch (1990) utilized a test-retest format to compare two elicitations of vowels, reading, and spontaneous speech samples on the dependent variables of mean fundamental frequency, standard deviation of fundamental frequency, and frequency perturbation or "jitter." Comparisons were made within gender groupings for 6 male and 6 female normal-speaking adult subjects. No significant differences were reported for test-retest means on any variable, and distribution statistics appear similar across repeated testing. By the end of the article the reader is left with the impression that there is a modicum of test-retest stability for all of the measures, but that it is maximized for the reading task "which had high test-retest consistency" (p. 362). Unfortunately, there are serious methodological problems with this study, which cast doubt on some reported findings and conclusions. Notably, there are two references to "correlation" in this article, but no correlation coefficients were reported. In the Abstract on page 360, the author states that "test-retest measures of perturbation using the Visi-Pitch did not have a high correlation coefficient." On p. 362 in the Discussion he further states that "reading ... yields a high test-retest correlation." The results section, however, is devoid of any report of correlation findings. This is particularly unfortunate in view of the fact that the correlation coefficient or similar reliability statistics (e.g., Rulon's formula) are uniquely appropriate for demonstrating the strength of association between two distributions in a test-retest study of this type (Guilford, 1965). Their absence is therefore puzzling and one wonders if they might not have been inadvertently deleted at some stage of the publication process. A potential problem of using parametric correlation procedures with such small sample sizes (n = 6) must also be acknowledged. In addition, reliability of subject performance is

TABLE 1. Hypothetical data set for male mean fundamental frequency on a reading task. Subject

Test

Retest A

Retest B

Retest C

1 2 3 4 5 6

93 101 107 113 118 146 113 18.41 93-146

91 99 111 115 120 142 113 17.77 91-142

142 120 115 111 99 91 113 17.77 91-142

111 99 142 91 120 115 113 17.77 91-142

X SD Range

1268

Downloaded From: https://jslhr.pubs.asha.org/ by a Universite Laval User on 03/27/2018 Terms of Use: https://pubs.asha.org/ss/rights_and_permissions.aspx

Letters Although systematic error is a separate issue from retest reliability (Horst, 1966), in support of Fitch's findings, it is valuable to know that the group dispersion statistics were replicable on repeated tests. It is possible to have two distributions demonstrate reliability, insofar as paired observations rise and fall together, yet also exhibit significantly different means. Systematic situations of this type exist when learning occurs as a result of experience with the testing instrument on the first administration. Setting aside considerations of limited sample size and power, Fitch has demonstrated that some aspects of normal group data using Visi-Pitch are consistent (i.e., replicable). Perhaps some explicit distinctions should have been made in his usage of the terms "consistency," "reliability," and "correlation," which appear to be undifferentiated in this article. In clinical practice we are more interested in the consistency (i.e., reliability) of individuals than of groups. Without correlational analyses of some type, their test-retest consistency remains unknown. Therefore, I concur with Fitch that "further studies ... on the measurements of the instrument are clearly warranted" (p. 363). Yet a potential danger of this study should be recognized, in that it may lead clinicians to infer that the Visi-Pitch is a reliable tool, when in fact that question cannot be answered by the incomplete research and statistical methodology that has been provided. Michael P. Cannito University of South Alabama Mobile, AL References Fitch, J. L. (1990). Consistency of fundamental frequency and perturbation in repeated phonations of sustained vowels, reading, and connected speech. Journal of Speech and Hearing Disorders, 55, 360-363. Guilford, J. P. (1965). Fundamental statistics in psychology and education. New York: McGraw Hill. Horst, P. (1966). Psychological measurement and prediction. Belmont, CA: Wadsworth.

1269

TABLE 1.Test-rtest correlations of fundamental frequency and perturbation for 12 subjects (6 male and 6 female). Speech sample Fundamental Vowels Reading Spontaneous speech Combined Perturbation Vowels

Male subjects

Female subjects

Cor

Prob

Corr

Prob

.8503 .8488 .7114

.0001 .0001 .0001

.7425 .8002 .4978

.0001 .0001 .0008

.8487

.0001

.8596

.0001

.4479

.0282

.4821

.0170

Vowel Production, Reading and Spontaneous Speech." The title change was recommended by the reviewers. I understood the reviewers' concerns and concurred with them. The final use of the terms "consistency" and "reliability" in the manuscript were at the suggestion of the reviewers. Also, the original manuscript contained test-retest correlations (see Table 1 below). The reviewers recommended that the table be removed. Although the above table may be useful, it should be interpreted cautiously. As one reviewer whose recommendations contributed to the removal of the table stated, "Correlation is [still] not an appropriate measure of reliability. Two sets of data can correlate very highly, but still be quite different." The consensus of the reviewers appeared to be that the basic statistical model published in the final manuscript was sufficient as a measure of consistency. I hope Cannito and others who have contacted me about various aspects of this study will continue their interest and enthusiasm for validating objective measures of acts of communication. Our credibility as a field can be greatly enhanced by tools that permit objective, reliable, and valid measures of communication in the clinical setting. We are still a fair distance from achieving that goal. James L. Fitch Auburn University

Received June 4, 1991 Accepted April 28, 1992

Received November 26, 1991 Accepted April 28, 1992

Response to Cannito I appreciate Cannito's insightful examination of my study and am in agreement with him on all points. This particular manuscript was subjected to a rather thorough review (there were three resubmissions) and the manuscript examined by the staff at Kay Elemetrics for accuracy. I feel the care with it was reviewed was warranted, realizing that the Visi-Pitch is a widely used instrument. The original title was "The Reliability and Consistency of Fundamental Frequency and Perturbation Measurement in

Attitudes Toward Nonspeaking Individuals Who Use Communication Boards In the February 1991 issue of JSHR, Gorenflo and Gorenflo reported that ". . . attitudes are significantly more favorable toward an individual using a technological augmentative communication technique such as a VOCA" (p. 23) than

Downloaded From: https://jslhr.pubs.asha.org/ by a Universite Laval User on 03/27/2018 Terms of Use: https://pubs.asha.org/ss/rights_and_permissions.aspx

1270 Journal of Speech and HearingResearch

35

000-000

December

992

TABLE 1. The percentage of the subjects who rated the alphabet board scenario (AB) and the communication board scenario (CB) who agreed with each statement. Statements

AB

CB

I would study (for a class) with this person. I respect this person. I feel sorry for this person. This person is not intelligent. This person would be easy to talk to. This person is not capable of giving a short speech to a class. I would help this person with a task such as purchasing something. This person is trustworthy. This person won't make a contribution to society. This person understands what people say. This person is sociable. I do not feel any sympathy for this person. I would feel uncomfortable with this person. I would feel inhibited with this person. I would not trust this person. This type of person would not be able to complete high school. You should not expect too much from this person. This person is as self-confident as other people. I would help this person obtain someone's attention. I would feel anxious around this person. This person would be successful ina job. This type of person is mentally handicapped. This person should expect to lead a normal life. I would feel uncomfortable answering questions asked by this person. This person would be able to complete college. This person is independent. I would prefer not to talk with this person. This person doesn't have a good social life.

56 97 38 9 12 6 88 53 9 72 38 0 34 25 0 0 6 34 72 44 12 25 16 16 56 34 34 3

70 95 14 0 14 0 97 57 5 78 46 8 24 14 0 0 3 32 86 35 51 10 32 8 92 48 5 8

toward one using a communication board on which only the alphabet appears. This result is not surprising considering the length of time it takes to communicate when only the alphabet is used. Most communication boards have common words and phrases on them in addition to the alphabet. Because communication with such a board would be less time-consuming than with a board with only an alphabet, we would expect attitudes of listeners toward individuals using the former type to be more favorable then toward those using the latter. To test this hypothesis, we evaluated the attitudes of 69 university students toward an individual using a communication board that had only the alphabet and one that had also had common words and phrases, using the scale in Appendix B of the Gorenflo and Gorenflo paper (see Table). The students were randomly assigned to two groups of approximately equal size. Both groups received the description of the individual from Appendix A of the Gorenflo and Gorenflo paper and a photograph of a communication board. Those in Group 1 received a photograph of a communication board that contained common words and phrases in addition to the alphabet (see Figure 5.6 in Silverman, 1989). Those in Group 2 received an enlarged photograph of the alphabet portion of this board. Analyses of the students' responses to the 28 items in the scale indicated that they generally had a more favorable attitude toward the individual when he used a communication board that contained common words and phrases in addition to the alphabet (see Table 1). For 25 of the 28 items, a higher percentage of the subjects in Group 1 than in Group 2 indicated a favorable attitude toward the individual. This difference was significant at the 0.01 level of

confidence (sign test). Although the average ratings of the subjects in Group 2 indicated they would prefer not to talk to the person when he used an alphabet board, those of the subjects in Group 1 indicated no such tendency. Thirty of the 37 subjects disagreed with the statement "I would prefer not to talk with this person." Furthermore, the individual was regarded as more independent and better able to complete college and be successful at work when he used the full communication board rather than just the alphabet portion. The results of our study are consistent with Gorenflo and Gorenflo's hypothesis that attitude favorability increases "... with the sophistication of the augmentative communication technique" (p. 19). A communication board that contains common words and phrases in addition to the alphabet obviously would be more sophisticated than one that contains only the alphabet. Although the favorability ratings for the VOCA probably would still be higher than those for the full communication board, the difference would be smaller than the one they reported between the VOCA and alphabet board. Whether the remaining difference would be large enough to result in a meaningful difference in attitudes toward individuals using these two communication techniques is uncertain.

Downloaded From: https://jslhr.pubs.asha.org/ by a Universite Laval User on 03/27/2018 Terms of Use: https://pubs.asha.org/ss/rights_and_permissions.aspx

Colleen Raney Marquette University Milwaukee, WI Franklin H. Silverman Marquette University Milwaukee, WI

Letters

1271

Silverman, F. H. (1989). Communication for the speechless (2nd ed.). Englewood Cliffs, NJ: Prentice Hall.

References Gorenflo, C. W., & Gorenflo, D.W. (1991). The effects of information and augmentative communication technique on attitudes toward nonspeaking individuals. Journal of Speech and Hearing Research, 34, 19-26.

Received May 26, 1992 Accepted June 9, 1992

Statement of Ownernhip Management end Circulation

fRehqukd by 39 U.S.C. 3685 4 P000w

IA

1.

Journal of Speech andHering Research (JSR)

PUBLICATIN NO

41

2 2 -

3.

P

3A. NO.o N..0 M Il

edAsn-

K-

C__^bdnO

. C*n1sM

aMbovM 4

See

S

Fw_ Pw5

Vn

_AWf81e *

HM

n

d

10801 Rockville Pike,

MOM Add d_ H -

d CiwM M_

W Mmqkq seM

PubiM. 6dM.

10801 Rockville Pike, Rockville,

A-sl.i4n

,.nrQ S. Sn n 0.000M0.If 0r... -

b

M )

.

·M

A r,.

Hon

nl

Inornt

It·*

5vit-

1Nn

I Pev

Fd edWh)., BOond

O

0

60

Qt

S. _1

STI

Ad

dI o0.

F. ceeo NMD0.W0 . oa u.l s . o-.. . 0TAL e,

O TOTALOf.

.

II (wrty -

36,618

&,

Pe oe

ht

waee t eotIeat

.e oe

mede by Wef loUby

end

ol

101 828.Jn~oy P. Pon 3626, ect IsQ

43,980 ---

42

- A-_ .

NO Coed.

67,914

18

67,956

36,636

3,595

7,254

w--_1

c

36,618

M

. mom

77Sn

67,914

C

I0.

m sn

Di

---

e-eb-.A

E. F7 _d -2qA.nn-p

?

.

71,551

E Ioo.MMObM.eO-`nfC0DD l

2

al

N. Cdte

e Ck.ebn n

by M.N CrM/ n 00

Obd,

I

WCAnl

nd- o

Co. n. a,

Y MOe._,

ATid.

C.gt1.

__.___

Me dM/I f3 los ; M o,.Nn

n

MeM 01T.M A-no of Bo Od o.

... ' PI...

2. MNI cTvtI C

..

?n-2.377Q

_

A. T1.l N. Cerd I Of. *. .04

SO.

20852-

hEse 0

EllX nd004

H4hq

^I

"

Et

rbkl.f11- Plki

e

t

J

ILL

MM"

ID01

tConib

rn I l....

N

I0

OnD

(es

trgngo and UIsa b

B

ea.t ffo. p?..f4

nrornLprtt.rnfolntd

-

D

of Coumnicative Disorders,

Department

IsCm

Slh-l.n.gis-De.rrAll.g Oricna

B F C

20852-3279

Ads,-)

Peesees

M

D

n-MUST NOrk.616

Terry L. Wiley, Ph.D., University of isconin-Hadison, adison, WI 53706 1975 illao Drive, Ma Edl. WWIM C-&. LA". Adb-I

(

Rockville,

bove

dCp CGpi

m

Fobs

(US) $126 (Foreign)

Orhl

American Speech-Lngusse-Hearing Association, 1I

at

oW_

,rjMn-

Aerican Speech-Language-Uearing Association,

s FNII

5 Annl 00w0.

In3v -d$114 Six issues

Si monthly

0

2

61 8 5 Sept. 30, 1992

--Al

71,551

. Pd4 .. 3 IIWO O d 104hile W icndT T. MA. Friderick SpahrPM. Ph.., WF

.0imd-,,

ao,

noe --. .

Downloaded From: https://jslhr.pubs.asha.org/ by a Universite Laval User on 03/27/2018 Terms of Use: https://pubs.asha.org/ss/rights_and_permissions.aspx

-)

....

.,.-

43,980 . KU-W. nOn-_ M~ e. Ok xecutive Director

eClsl ICe..

Attitudes toward nonspeaking individuals who use communication boards.

Letters to the Editor A Questionable Consistency: Response to Fitch confounded in this study with reliability of the instrument itself, which should...
347KB Sizes 0 Downloads 0 Views