This article was downloaded by: [University of Birmingham] On: 07 January 2015, At: 22:17 Publisher: Routledge Informa Ltd Registered in England and Wales Registered Number: 1072954 Registered office: Mortimer House, 37-41 Mortimer Street, London W1T 3JH, UK
Research Quarterly for Exercise and Sport Publication details, including instructions for authors and subscription information: http://www.tandfonline.com/loi/urqe20
Establishing Reliability of Biomechanical Data Using Univariate and Multivariate Approaches a
b
Marilyn A. Looney , Sarah L. Smith & Syamala Srinivasan
c
a
Department of Physical Education , Northern Illinois University , DeKalb , IL , USA
b
Sports Science Division , United States Olympic Committee , Colorado Springs , CO , USA
c
Statistician with Nalco Chemical Company , Naperville , IL , USA Published online: 26 Feb 2013.
To cite this article: Marilyn A. Looney , Sarah L. Smith & Syamala Srinivasan (1990) Establishing Reliability of Biomechanical Data Using Univariate and Multivariate Approaches, Research Quarterly for Exercise and Sport, 61:2, 154-161, DOI: 10.1080/02701367.1990.10608669 To link to this article: http://dx.doi.org/10.1080/02701367.1990.10608669
PLEASE SCROLL DOWN FOR ARTICLE Taylor & Francis makes every effort to ensure the accuracy of all the information (the “Content”) contained in the publications on our platform. However, Taylor & Francis, our agents, and our licensors make no representations or warranties whatsoever as to the accuracy, completeness, or suitability for any purpose of the Content. Any opinions and views expressed in this publication are the opinions and views of the authors, and are not the views of or endorsed by Taylor & Francis. The accuracy of the Content should not be relied upon and should be independently verified with primary sources of information. Taylor and Francis shall not be liable for any losses, actions, claims, proceedings, demands, costs, expenses, damages, and other liabilities whatsoever or howsoever caused arising directly or indirectly in connection with, in relation to or arising out of the use of the Content. This article may be used for research, teaching, and private study purposes. Any substantial or systematic reproduction, redistribution, reselling, loan, sub-licensing, systematic supply, or distribution in any form to anyone is expressly forbidden. Terms & Conditions of access and use can be found at http:// www.tandfonline.com/page/terms-and-conditions
LooNEY, SMITH, AND SRINIVASAN
RESEARCH QUARTBRLY
FOR EXBRCISB AND SPORT
1990, VOl.. 61, No.2, pp. 154-160
Establishing Reliability of Biomechanical Data Using Univariate and Multivariate Approaches MARILYN A. LOONEY Northern Illinois University
SARAHL. SMITH
Downloaded by [University of Birmingham] at 22:17 07 January 2015
U.S. Olympic Committee, Colorado Springs, Colorado SYAMALA SRINIVASAN Naperville, Illinois
The purpose ofthis study was to use generalizability theory with both univariate and multivariate approaches to examine reliability oftotal body center ofmass (CM) values calculatedfrom cinematographical daia. Twenty-eight college-aged male volunteers were filmed by a WCAM camera at 100 fps while performing the basic locomotion skill ofwalking. Film analysis was conducted on eacn subject using six frames offilm depicting a one-stride walking cycle consisting ofright heel strike, right foot flat, left toe-off, left heel strike, left foot flat, and right toe-off. Nineteen segmental endpoints and a reference point were digitized by three experienced plotters. The digitizing sequence was replicated three times by eacn plotter. A FORTRAN program calculated nine CM values (three plotters by three repetitions) for eacb subject filmed in eacb ofthe six positions ofthe stride. The x- and y-coordinates ofthe CM values were the dependent variables analyzed by fully crossed 3-way univariate and multivariate ANOVAs (subjects by plotters by repetitions). All measuremenifacets were considered to be random. Results indicate that there was very little repetition error but considerable interplotter error for most frames. Phi-coefficients for x- and y-coordinates, separately,fluctuated across frames. The univariate values for the x-coordinates were similar but slightly less than the multivariate values. The Phi-coefficients for Y, however, were considerably lower than the multivariate values. The multivariate Phi-coefficients for generalizing over three plotters and three repetitions rangedfrom .82 to .90. When restricted to the typical protocol ofone person digitizing each. anatomical landmark once, Phi-coefficients ranged from 56 to .74./t was concluded that multivariate approaches should be used to determine reliability of Cartesian coordinates and that reliability ofCM is not consistent across afilm sequence. The inability ofplotters to identify hidden landmarks and plotter fatigue and boredom may have been major contributors to measurement imprecision.
Keywords:reliability,biomechanics,walking,centerof mass
The
purposeof this study was to use generalizability theory withboth univariateand multivariateapproachesto examine RBsBAROI QuAR1llRLY PaR
the reliability of total body center of mass (CM) values calculated from cinematographical data. A restriction imposedonthestudywasthatthedatacollectionphasebesimilar andapplicabletothe"real-world"situationfacingbiomechanics researchers. In sport biomechanics research, each of the calculatedCM values is consideredto be the most representative of the body position assumed by an athlete during the executionofa motorskillat a specificpoint intime.TheseCM valuesare determinedusingthe segmentalmethodapproach; the resulting displacement, velocity, and acceleration data providedescriptiveinformation regardingthespecificmovement being analyzed. Despite the importance given to CM calculationsin theanalysisof sportskills,mostbiomechanics researchers fail to provideinformation aboutthe reliability of the CM data being presented and interpreted. Disch and Hudson (1980) have stated that "the reliability questions importantto thebiomechanics researcherare ones of stability and objectivity," but "unfortunately,this importantmeasurement phase of biomechanical research is often overlooked" (p.196). In the few instances in which reliability data have been reported, intraelass correlation coefficients were calculated for repeated measures of the digitized segmental endpoints that are necessaryfor determiningCM values. Two or three digitizing trials were the usual practice observed in these studies (Barlow, 1973; Ward, 1973). The lowest reported reliabilitycorrelation coefficient was .91 for both the x- and y-coordinates; the highest reported value was .99. French (1981) investigatedthe effects of film emulsion, camera frame rate, and film/camera format (8 or 16 mm) on the eM displacementvaluesdeterminedforsubjectsperformingthe skillofrunning.In addition,intraplotterreliabilitywas determined. Intraclasscorrelationcoefficientsfor theabscissa ranged from .917 to .999; the corresponding values for the ordinatewere .906 to .999 (p. 61). French concludedthat the
EXBRasa
154
AND SPORT, VOL.
61, No.2
Downloaded by [University of Birmingham] at 22:17 07 January 2015
LooNEY, SMlTIl, AND SRINIVASAN
horizontal coordinate values were located with greater accuracy than were the vertical coordinates. A subproblem of the study conducted by Davis (1973) was to determine intraplotter reliability and objectivity of six plotters. Each plotter digitized three trials of the segmental endpoints of two subjects with differing anthropometric characteristics. Still photographs, with different image sizes, depicting the subjects in threebody positions were therecording medium. Deviation scores, obtained by subtracting a criterion CM value determined on a reaction board from the corresponding segmental method CM value, were used by Davis in the analysis. Intraclass coefficients computed for these x- andy-coordinate deviation scores were .968and .944, respectively. Davis could not compute an intraclass correlation coefficient for objectivityof the six plotters; however,the analysis of variance of the deviation scores indicated that agreement among plotters did not exist (pp. 95-96). In two other studies, the researchers reported that digitizingwasperformedthreetimesforall trials.Zemicke,Caldwell, and Roberts (1976) reported using the average of three digitized trials, but Roberts (1971) did not state if mean values were used. Both Bates (1973) and Disch and Hudson (1980) concurred with the practice of using the average of at least three digitizing trials of the segmental endpoint data before CM values are calculated. However, no quantitative analysis has been reported to verify that the average of three digitizing trials is required for the data to have acceptable reliability. Disch and Hudson acknowledged the time factor involved in digitizing three or more trials but emphasized that "if reliable data is [sic] to be obtained, the criterion of stability must be met" (p. 198). Although the digitizing procedures discussed above impact the reliability of the segmentalendpoint measuresand subsequent CM values, there are other inherent factors in two-dimensional cinematography that may contribute to decreasing the accuracy of film data. These potential sources of error are delineated in the studies of Snowden (1975) and McLaughlin, Dillman, and Lardner (1977). According to Snowden (1975),errors in cinematography can be classified into three categories. These errors are (a) due to the film, (b) in the projection system, and (c) in the collection of the data (pp. 3-5). Film causes asymmetrical image distortion with greater differences in the vertical than the horizontal direction. Triacetate film has dimensional instability that is not of a linear nature. Snowden noted that optical distortions in the projection system varies with lens quality and that this distortion can be five times greater at the periphery than at the center. Data collection errors arise from the resolution of the digitizing equipment and from human judgment errors that can be both systematic and random. Snowden also advocated taking the mean of several readings. In their investigation,McLaughlinet al. (1977)identified 10 sources of error inherent in the procedure used to record and measure sample data. These sources of error were as follows: (a) movement of segmental markers relative to axes
of rotation, (b) distortion introduced by optical system of camera, (c) movement of camera, (d) graininess of film, (e) camera position relative to plane of motion, (f) movementout of primary plane, (g) resolution of image, (h) stretching of film, (i) alignment of frames during the analysis, and (j) recording errors (p. 573). They also noted that many of these errors can be reduced to those of distortion and measurement procedures. In a more recent investigation, Smithand Looney (1985) examined previous methods used by biomechanics researchers to establish the reliability of center of gravity values and proposed the use of generalizability theory as a method for determining reliability estimates. They used film data obtained from 28 subjects (males = 14; females = 14) while walking as an example to illustrate the application of generalizability theoryin determining the reliability of different measurement procedures for obtaining center of gravity values. Their concern was that biomechanists do not address theissues of reliability,and hencevalidity,of the datareported in their research studies. Since many errors of distortion can be reduced by controllingfactors related to the filming setup, choice of film,and camera andprojectionequipment,attentionshould be focused on the measurementprocedures used to obtain biomechanical data.Few studies have attempted to quantify the magnitudeof the error associated with measurement procedures or to suggest the most appropriate measurement procedure for ensuring reliable cinematographical data. Earlier approaches to establishing reliability have been single-faceted-only intraplotter or interplotter error has been investigated. In addition, reliability has only been established for x- and y-coordinates separately for isolated frames. Thus, the reliability of a coordinate pair needs to be pursued for an entire film sequence. Researchers in biomechanics need to determine thereliability of thedata beingreported from their laboratories.Basedon the measurement procedure employed regarding such factors as image size, digitizing sequence, data smoothing techniques, and so on, biomechanists need to provide an appropriate assessment of the reliability of the data describing human performance. Because of the several sources of error associated with digitizing,a reliability techniqueshould be used that incorporates more than one source of error. Such a technique is the application of generalizability theory formulated by Cronbach, Gieser, Nanda, and Rajaratnam (1972).
Methods Twenty-eightcollege-agedmales volunteeredas subjects for this study.Average heightand weight valuesfor thisgroup were 177.57 em (SD = 8.83) and 72.76 kg (SD = 11.30). During the filming, each subject was attired in shorts and athletic shoes. Segmental landmarks were not marked because the filming procedure was to be representative of competitive situations.
REsEAROI QuARTIlRLY FOR ExBRCISB AND SPORT,
155
VOL. 61, No.2
Downloaded by [University of Birmingham] at 22:17 07 January 2015
LooNEY, SMITH, AND SRINIVASAN
Subjects performed the basic locomotor skill of walking as they were filmed by a LOCAM camera operating at 100 frames persecond. This skill was selectedfor analysis because of the planar motion involved. Black and white Kodak RAR 2498 reversal film was used. This Polyester-based film minimized the effects of dimensional instability associated with triacetate base film. Appropriate camera settings and film processing produced a high quality film with respect to image sharpness and contrast. Camera-to-subject distance was 12.19 m, which aided in the reduction of perspective error. In combination with a 25mm camera lens, a field of view of 4.88 m by 3.35 m was provided. Lens height during the filming was 1.3 m. Film analysis was conducted on each subject using six frames of film, which depicted a one-stride walking cycle consisting of right heel strike, right foot flat, left toe-off, left heel strike, left foot flat, and right toe-off. All six frames in each walking sequence occurred near the center of the film frame so that any optical distortion of the projection system would be minimized. The selected film frames were marked to ensure that identical frames would be digitized by the plotters. The film analysis system consisted of a Selecta-Frame 5 Bell and Howell 16mm Analysis Projector and a Numonics 1220 digitizing unit interfaced through an ADP computer terminal to an NAS-9000 mainframe computer. This system provided for the rear projection of the film ooto a digitizing surface; average image height for the subjects exceeded onehalf ofthe projected height ofthe film frame. Resolution ofthe digitizing unit was .25 mm. Nineteen segmental endpoints and a reference point in each of the six film frames selected for analysis were digitized by three experienced plotters. Although six frames were the focus ofthis study, three frames immediately prior to and after the six frames depicting the walking stride were also digitized. These frames were digitized for future studies to determine the influence of smoothing routines on biomechanical data. By increasing the number of frames digitized per subject from 6 to 12, the potential for plotter fatigue and/or boredom increased. Each of the 12 specified frames for a subject was successively digitized three times by each of the plotters. The subjects were arbitrarily placed into five groups, and plotters digitized these groups in a randomly selected order. Nine CM values (3 plotters by 3 repetitions) were calculated from the segmental endpoints by a FORTRAN computer program for each of the 28 subjects in the six designated positions of the walking stride. This program automatically aligned the digitized frames to correct for any misalignment during projection onto the digitizing surface. The program alsoprovided for all CM values to be located with respect to a common origin. The x- and y-coordinates of the CM measures (the dependent variables) were then analyzed by BMDPSV (BMDP Statistical Software, 1985) and ANOVA and MATRIX procedures (SAS Institute, 1985) using a fully crossed 28 x 3 x 3 (subjects by repetitions by plotters) ANOVA design. Both the RilsBAROi
repetition and plotter sources of measurement error, called facets in a generalizability study, were considered to be random. A separate univariate and multivariate generalizability (G) study was completed for each frame of the walking cycle. In a G-study the variance of subjects' scores based on the specified measurement conditions is partitioned. The magnitude ofthe individual variance components is then used to determine which source had the greatest influence on universe score variability. Four decision (D) studies were also conducted to determine how reliability was influenced when the number of plotters or repetitions was modified. In D-studies the universal generalization and/or the conceptualization of universe score interpretation (i.e., norm-referenced or domain-referenced) are manipulated. Reliability indices are determined for each of the different universes of generalization to help identify the best measurement protocol. In order to compute a reliability index, estimates of universe (true) score and error variance were required. Variance attributed to subjects was used as the estimate ofuniverse score variance and absolute error variance was used as the estimate oferror variance. Often in biomechanical studies the decision about an individual's performance is made without considering the performance of other people. Within this context a reliability index should reflect how consistently an individual's universe score is replicated and not how consistently a subject has the same relative position within the group ofsubjects measured. Thus, every variance componentexcept variance due to subjects is considered error. Absolute error variance (&/12) is the variance of the differences between the subjects' observed and universe scores and is defined as
+ -
+
+ -
+ -
+
where n is the number of levels for the identified facet over which one wishes to generalize (Brennan, 1984). The subscripts R,P, and S represent repetitions, plotters, and subjects terms, respectively. Any combination of subscripts (e.g.,RP) denotes the interaction of the specified terms. This notation will be used throughout the rest of the article. The reliability index for a D-study is called an index of dependability (c!» or Phi-coefficient when error is defined as absolute variance. It is denoted as
c!>
= ---=---
(Brennan, 1984).
In order to generate a multivariate index ofdependability, canonical coefficients were determined that maximize the ratio of universe score variation to the universe score plus absolute error variation. The index of dependability is calculated by using the following equation:
QuAR11lRLY POR ExBROSIl AND SPOIlT, VOL.
156
61, No.2
LooNBY. SMmI, AND SRINIVASAN
ell
=
q'Y,q + q~q + q'Vpq + q'VIrq + q'VSl/q + q'Vgq + q'Vwq nR n,. nh nR np nh
-----------
where V is a matrix of varianceand covariancecomponents, n is thenumberof levelsfor theidentifiedfacetin thedecision
study, and 9 is the vector of canonical coefficients. The reportedindicesof dependability arebased on the firstcanonical variate (Webb, Shavelson, & Maddahian, 1983).
Downloaded by [University of Birmingham] at 22:17 07 January 2015
Results The varianceestimatesderivedfrom the generalizability studiesarepresentedin Table 1.They seem to followa similar pattern across frames; however, the percentages of total variation accounted for by each variance component (see Table 2) clarify the pattern.
Thevariationamongsubjectsaccountsforthemajority of variationacrossframesfor thez-coordinate(48.8%t074.1%). whereasthis is nottruefor thej-coordinate (21.1% to 32.1%). No variability (
Research Quarterly for Exercise and Sport Publication details, including instructions for authors and subscription information: http://www.tandfonline.com/loi/urqe20
Establishing Reliability of Biomechanical Data Using Univariate and Multivariate Approaches a
b
Marilyn A. Looney , Sarah L. Smith & Syamala Srinivasan
c
a
Department of Physical Education , Northern Illinois University , DeKalb , IL , USA
b
Sports Science Division , United States Olympic Committee , Colorado Springs , CO , USA
c
Statistician with Nalco Chemical Company , Naperville , IL , USA Published online: 26 Feb 2013.
To cite this article: Marilyn A. Looney , Sarah L. Smith & Syamala Srinivasan (1990) Establishing Reliability of Biomechanical Data Using Univariate and Multivariate Approaches, Research Quarterly for Exercise and Sport, 61:2, 154-161, DOI: 10.1080/02701367.1990.10608669 To link to this article: http://dx.doi.org/10.1080/02701367.1990.10608669
PLEASE SCROLL DOWN FOR ARTICLE Taylor & Francis makes every effort to ensure the accuracy of all the information (the “Content”) contained in the publications on our platform. However, Taylor & Francis, our agents, and our licensors make no representations or warranties whatsoever as to the accuracy, completeness, or suitability for any purpose of the Content. Any opinions and views expressed in this publication are the opinions and views of the authors, and are not the views of or endorsed by Taylor & Francis. The accuracy of the Content should not be relied upon and should be independently verified with primary sources of information. Taylor and Francis shall not be liable for any losses, actions, claims, proceedings, demands, costs, expenses, damages, and other liabilities whatsoever or howsoever caused arising directly or indirectly in connection with, in relation to or arising out of the use of the Content. This article may be used for research, teaching, and private study purposes. Any substantial or systematic reproduction, redistribution, reselling, loan, sub-licensing, systematic supply, or distribution in any form to anyone is expressly forbidden. Terms & Conditions of access and use can be found at http:// www.tandfonline.com/page/terms-and-conditions
LooNEY, SMITH, AND SRINIVASAN
RESEARCH QUARTBRLY
FOR EXBRCISB AND SPORT
1990, VOl.. 61, No.2, pp. 154-160
Establishing Reliability of Biomechanical Data Using Univariate and Multivariate Approaches MARILYN A. LOONEY Northern Illinois University
SARAHL. SMITH
Downloaded by [University of Birmingham] at 22:17 07 January 2015
U.S. Olympic Committee, Colorado Springs, Colorado SYAMALA SRINIVASAN Naperville, Illinois
The purpose ofthis study was to use generalizability theory with both univariate and multivariate approaches to examine reliability oftotal body center ofmass (CM) values calculatedfrom cinematographical daia. Twenty-eight college-aged male volunteers were filmed by a WCAM camera at 100 fps while performing the basic locomotion skill ofwalking. Film analysis was conducted on eacn subject using six frames offilm depicting a one-stride walking cycle consisting ofright heel strike, right foot flat, left toe-off, left heel strike, left foot flat, and right toe-off. Nineteen segmental endpoints and a reference point were digitized by three experienced plotters. The digitizing sequence was replicated three times by eacn plotter. A FORTRAN program calculated nine CM values (three plotters by three repetitions) for eacb subject filmed in eacb ofthe six positions ofthe stride. The x- and y-coordinates ofthe CM values were the dependent variables analyzed by fully crossed 3-way univariate and multivariate ANOVAs (subjects by plotters by repetitions). All measuremenifacets were considered to be random. Results indicate that there was very little repetition error but considerable interplotter error for most frames. Phi-coefficients for x- and y-coordinates, separately,fluctuated across frames. The univariate values for the x-coordinates were similar but slightly less than the multivariate values. The Phi-coefficients for Y, however, were considerably lower than the multivariate values. The multivariate Phi-coefficients for generalizing over three plotters and three repetitions rangedfrom .82 to .90. When restricted to the typical protocol ofone person digitizing each. anatomical landmark once, Phi-coefficients ranged from 56 to .74./t was concluded that multivariate approaches should be used to determine reliability of Cartesian coordinates and that reliability ofCM is not consistent across afilm sequence. The inability ofplotters to identify hidden landmarks and plotter fatigue and boredom may have been major contributors to measurement imprecision.
Keywords:reliability,biomechanics,walking,centerof mass
The
purposeof this study was to use generalizability theory withboth univariateand multivariateapproachesto examine RBsBAROI QuAR1llRLY PaR
the reliability of total body center of mass (CM) values calculated from cinematographical data. A restriction imposedonthestudywasthatthedatacollectionphasebesimilar andapplicabletothe"real-world"situationfacingbiomechanics researchers. In sport biomechanics research, each of the calculatedCM values is consideredto be the most representative of the body position assumed by an athlete during the executionofa motorskillat a specificpoint intime.TheseCM valuesare determinedusingthe segmentalmethodapproach; the resulting displacement, velocity, and acceleration data providedescriptiveinformation regardingthespecificmovement being analyzed. Despite the importance given to CM calculationsin theanalysisof sportskills,mostbiomechanics researchers fail to provideinformation aboutthe reliability of the CM data being presented and interpreted. Disch and Hudson (1980) have stated that "the reliability questions importantto thebiomechanics researcherare ones of stability and objectivity," but "unfortunately,this importantmeasurement phase of biomechanical research is often overlooked" (p.196). In the few instances in which reliability data have been reported, intraelass correlation coefficients were calculated for repeated measures of the digitized segmental endpoints that are necessaryfor determiningCM values. Two or three digitizing trials were the usual practice observed in these studies (Barlow, 1973; Ward, 1973). The lowest reported reliabilitycorrelation coefficient was .91 for both the x- and y-coordinates; the highest reported value was .99. French (1981) investigatedthe effects of film emulsion, camera frame rate, and film/camera format (8 or 16 mm) on the eM displacementvaluesdeterminedforsubjectsperformingthe skillofrunning.In addition,intraplotterreliabilitywas determined. Intraclasscorrelationcoefficientsfor theabscissa ranged from .917 to .999; the corresponding values for the ordinatewere .906 to .999 (p. 61). French concludedthat the
EXBRasa
154
AND SPORT, VOL.
61, No.2
Downloaded by [University of Birmingham] at 22:17 07 January 2015
LooNEY, SMlTIl, AND SRINIVASAN
horizontal coordinate values were located with greater accuracy than were the vertical coordinates. A subproblem of the study conducted by Davis (1973) was to determine intraplotter reliability and objectivity of six plotters. Each plotter digitized three trials of the segmental endpoints of two subjects with differing anthropometric characteristics. Still photographs, with different image sizes, depicting the subjects in threebody positions were therecording medium. Deviation scores, obtained by subtracting a criterion CM value determined on a reaction board from the corresponding segmental method CM value, were used by Davis in the analysis. Intraclass coefficients computed for these x- andy-coordinate deviation scores were .968and .944, respectively. Davis could not compute an intraclass correlation coefficient for objectivityof the six plotters; however,the analysis of variance of the deviation scores indicated that agreement among plotters did not exist (pp. 95-96). In two other studies, the researchers reported that digitizingwasperformedthreetimesforall trials.Zemicke,Caldwell, and Roberts (1976) reported using the average of three digitized trials, but Roberts (1971) did not state if mean values were used. Both Bates (1973) and Disch and Hudson (1980) concurred with the practice of using the average of at least three digitizing trials of the segmental endpoint data before CM values are calculated. However, no quantitative analysis has been reported to verify that the average of three digitizing trials is required for the data to have acceptable reliability. Disch and Hudson acknowledged the time factor involved in digitizing three or more trials but emphasized that "if reliable data is [sic] to be obtained, the criterion of stability must be met" (p. 198). Although the digitizing procedures discussed above impact the reliability of the segmentalendpoint measuresand subsequent CM values, there are other inherent factors in two-dimensional cinematography that may contribute to decreasing the accuracy of film data. These potential sources of error are delineated in the studies of Snowden (1975) and McLaughlin, Dillman, and Lardner (1977). According to Snowden (1975),errors in cinematography can be classified into three categories. These errors are (a) due to the film, (b) in the projection system, and (c) in the collection of the data (pp. 3-5). Film causes asymmetrical image distortion with greater differences in the vertical than the horizontal direction. Triacetate film has dimensional instability that is not of a linear nature. Snowden noted that optical distortions in the projection system varies with lens quality and that this distortion can be five times greater at the periphery than at the center. Data collection errors arise from the resolution of the digitizing equipment and from human judgment errors that can be both systematic and random. Snowden also advocated taking the mean of several readings. In their investigation,McLaughlinet al. (1977)identified 10 sources of error inherent in the procedure used to record and measure sample data. These sources of error were as follows: (a) movement of segmental markers relative to axes
of rotation, (b) distortion introduced by optical system of camera, (c) movement of camera, (d) graininess of film, (e) camera position relative to plane of motion, (f) movementout of primary plane, (g) resolution of image, (h) stretching of film, (i) alignment of frames during the analysis, and (j) recording errors (p. 573). They also noted that many of these errors can be reduced to those of distortion and measurement procedures. In a more recent investigation, Smithand Looney (1985) examined previous methods used by biomechanics researchers to establish the reliability of center of gravity values and proposed the use of generalizability theory as a method for determining reliability estimates. They used film data obtained from 28 subjects (males = 14; females = 14) while walking as an example to illustrate the application of generalizability theoryin determining the reliability of different measurement procedures for obtaining center of gravity values. Their concern was that biomechanists do not address theissues of reliability,and hencevalidity,of the datareported in their research studies. Since many errors of distortion can be reduced by controllingfactors related to the filming setup, choice of film,and camera andprojectionequipment,attentionshould be focused on the measurementprocedures used to obtain biomechanical data.Few studies have attempted to quantify the magnitudeof the error associated with measurement procedures or to suggest the most appropriate measurement procedure for ensuring reliable cinematographical data. Earlier approaches to establishing reliability have been single-faceted-only intraplotter or interplotter error has been investigated. In addition, reliability has only been established for x- and y-coordinates separately for isolated frames. Thus, the reliability of a coordinate pair needs to be pursued for an entire film sequence. Researchers in biomechanics need to determine thereliability of thedata beingreported from their laboratories.Basedon the measurement procedure employed regarding such factors as image size, digitizing sequence, data smoothing techniques, and so on, biomechanists need to provide an appropriate assessment of the reliability of the data describing human performance. Because of the several sources of error associated with digitizing,a reliability techniqueshould be used that incorporates more than one source of error. Such a technique is the application of generalizability theory formulated by Cronbach, Gieser, Nanda, and Rajaratnam (1972).
Methods Twenty-eightcollege-agedmales volunteeredas subjects for this study.Average heightand weight valuesfor thisgroup were 177.57 em (SD = 8.83) and 72.76 kg (SD = 11.30). During the filming, each subject was attired in shorts and athletic shoes. Segmental landmarks were not marked because the filming procedure was to be representative of competitive situations.
REsEAROI QuARTIlRLY FOR ExBRCISB AND SPORT,
155
VOL. 61, No.2
Downloaded by [University of Birmingham] at 22:17 07 January 2015
LooNEY, SMITH, AND SRINIVASAN
Subjects performed the basic locomotor skill of walking as they were filmed by a LOCAM camera operating at 100 frames persecond. This skill was selectedfor analysis because of the planar motion involved. Black and white Kodak RAR 2498 reversal film was used. This Polyester-based film minimized the effects of dimensional instability associated with triacetate base film. Appropriate camera settings and film processing produced a high quality film with respect to image sharpness and contrast. Camera-to-subject distance was 12.19 m, which aided in the reduction of perspective error. In combination with a 25mm camera lens, a field of view of 4.88 m by 3.35 m was provided. Lens height during the filming was 1.3 m. Film analysis was conducted on each subject using six frames of film, which depicted a one-stride walking cycle consisting of right heel strike, right foot flat, left toe-off, left heel strike, left foot flat, and right toe-off. All six frames in each walking sequence occurred near the center of the film frame so that any optical distortion of the projection system would be minimized. The selected film frames were marked to ensure that identical frames would be digitized by the plotters. The film analysis system consisted of a Selecta-Frame 5 Bell and Howell 16mm Analysis Projector and a Numonics 1220 digitizing unit interfaced through an ADP computer terminal to an NAS-9000 mainframe computer. This system provided for the rear projection of the film ooto a digitizing surface; average image height for the subjects exceeded onehalf ofthe projected height ofthe film frame. Resolution ofthe digitizing unit was .25 mm. Nineteen segmental endpoints and a reference point in each of the six film frames selected for analysis were digitized by three experienced plotters. Although six frames were the focus ofthis study, three frames immediately prior to and after the six frames depicting the walking stride were also digitized. These frames were digitized for future studies to determine the influence of smoothing routines on biomechanical data. By increasing the number of frames digitized per subject from 6 to 12, the potential for plotter fatigue and/or boredom increased. Each of the 12 specified frames for a subject was successively digitized three times by each of the plotters. The subjects were arbitrarily placed into five groups, and plotters digitized these groups in a randomly selected order. Nine CM values (3 plotters by 3 repetitions) were calculated from the segmental endpoints by a FORTRAN computer program for each of the 28 subjects in the six designated positions of the walking stride. This program automatically aligned the digitized frames to correct for any misalignment during projection onto the digitizing surface. The program alsoprovided for all CM values to be located with respect to a common origin. The x- and y-coordinates of the CM measures (the dependent variables) were then analyzed by BMDPSV (BMDP Statistical Software, 1985) and ANOVA and MATRIX procedures (SAS Institute, 1985) using a fully crossed 28 x 3 x 3 (subjects by repetitions by plotters) ANOVA design. Both the RilsBAROi
repetition and plotter sources of measurement error, called facets in a generalizability study, were considered to be random. A separate univariate and multivariate generalizability (G) study was completed for each frame of the walking cycle. In a G-study the variance of subjects' scores based on the specified measurement conditions is partitioned. The magnitude ofthe individual variance components is then used to determine which source had the greatest influence on universe score variability. Four decision (D) studies were also conducted to determine how reliability was influenced when the number of plotters or repetitions was modified. In D-studies the universal generalization and/or the conceptualization of universe score interpretation (i.e., norm-referenced or domain-referenced) are manipulated. Reliability indices are determined for each of the different universes of generalization to help identify the best measurement protocol. In order to compute a reliability index, estimates of universe (true) score and error variance were required. Variance attributed to subjects was used as the estimate ofuniverse score variance and absolute error variance was used as the estimate oferror variance. Often in biomechanical studies the decision about an individual's performance is made without considering the performance of other people. Within this context a reliability index should reflect how consistently an individual's universe score is replicated and not how consistently a subject has the same relative position within the group ofsubjects measured. Thus, every variance componentexcept variance due to subjects is considered error. Absolute error variance (&/12) is the variance of the differences between the subjects' observed and universe scores and is defined as
+ -
+
+ -
+ -
+
where n is the number of levels for the identified facet over which one wishes to generalize (Brennan, 1984). The subscripts R,P, and S represent repetitions, plotters, and subjects terms, respectively. Any combination of subscripts (e.g.,RP) denotes the interaction of the specified terms. This notation will be used throughout the rest of the article. The reliability index for a D-study is called an index of dependability (c!» or Phi-coefficient when error is defined as absolute variance. It is denoted as
c!>
= ---=---
(Brennan, 1984).
In order to generate a multivariate index ofdependability, canonical coefficients were determined that maximize the ratio of universe score variation to the universe score plus absolute error variation. The index of dependability is calculated by using the following equation:
QuAR11lRLY POR ExBROSIl AND SPOIlT, VOL.
156
61, No.2
LooNBY. SMmI, AND SRINIVASAN
ell
=
q'Y,q + q~q + q'Vpq + q'VIrq + q'VSl/q + q'Vgq + q'Vwq nR n,. nh nR np nh
-----------
where V is a matrix of varianceand covariancecomponents, n is thenumberof levelsfor theidentifiedfacetin thedecision
study, and 9 is the vector of canonical coefficients. The reportedindicesof dependability arebased on the firstcanonical variate (Webb, Shavelson, & Maddahian, 1983).
Downloaded by [University of Birmingham] at 22:17 07 January 2015
Results The varianceestimatesderivedfrom the generalizability studiesarepresentedin Table 1.They seem to followa similar pattern across frames; however, the percentages of total variation accounted for by each variance component (see Table 2) clarify the pattern.
Thevariationamongsubjectsaccountsforthemajority of variationacrossframesfor thez-coordinate(48.8%t074.1%). whereasthis is nottruefor thej-coordinate (21.1% to 32.1%). No variability (
