Reliability and Validity of a Short Version of the General Functioning Subscale of the McMaster Family Assessment Device KATRINA L. BOTERHOVEN DE HAAN* JENNIFER HAFEKOST* DAVID LAWRENCE* MICHAEL G. SAWYER† STEPHEN R. ZUBRICK*

The General Functioning 12-item subscale (GF12) of The McMaster Family Assessment Device (FAD) has been validated as a single index measure to assess family functioning. This study reports on the reliability and validity of using only the six positive items from the General Functioning subscale (GF6+). Existing data from two Western Australian studies, the Raine Study (RS) and the Western Australian Child Health Survey (WACHS), was used to analyze the psychometric properties of the GF6+ subscale. The results demonstrated that the GF6+ subscale had virtually equivalent psychometric properties and was able to identify almost all of the same families who had healthy or unhealthy levels of functioning as the full GF12 subscale. In consideration of the constraints faced by large-scale population-based surveys, the findings of this study support the use of a GF6+ subscale from the FAD, as a quick and effective tool to assess the overall functioning of families. Keywords: Family Assessment Device; McMaster Model; Family Functioning; General Functioning; Bradburn Scale; Psychometrics Fam Proc 54:116–123, 2015

L

arge-scale population-based surveys are challenged with capturing data on a wide range of domains within the constraints of a fixed interview length (Kessler et al., 2002). The choice of measures used in these surveys is often determined on the basis of suitability, respondent and interviewer burden, and ease of administration, as well as the psychometric properties of the measure (Kessler et al., 2003; Sharp & Frankel, 1983). In a study where family functioning is the main focus of investigation, use of a detailed measure, such as the McMaster Family Assessment Device (FAD), may be appropriate (Epstein, Baldwin, & Bishop, 1983). In surveys where family functioning is not the main focus, but provides important contextual information, a shorter instrument may be desired. This has led to the more frequent use of the General Functioning (GF) subscale of the FAD in population-based surveys rather than the longer, complete FAD (Byles, Byrne, Boyle, & Offord, 1988). As a general principle, abbreviated versions of scales which

*Telethon Kids Institute, The University of Western Australia, Perth, WA, Australia. † Research and Evaluation Unit, Women’s and Children’s Health Network, Discipline of Paediatrics, University of Adelaide, Adelaide, SA, Australia.

Correspondence concerning this article should be addressed to Katrina Boterhoven de Haan, Telethon Kids Institute, The University of Western Australia, PO Box 855, West Perth, WA 6872, Australia. E-mail: [email protected]. Thanks to John Ainley for the helpful comments and advice on the manuscript. 116

Family Process, Vol. 54, No. 1, 2015 © 2014 Family Process Institute doi: 10.1111/famp.12113

/ 117

BOTERHOVEN DE HAAN, HAFEKOST, LAWRENCE, SAWYER, & ZUBRICK

correlate highly with longer versions of instruments provide the opportunity of capturing information that would otherwise not be possible to include (Kessler et al., 2003). The FAD is a 60-item self-report measure which operationalizes the McMaster Model of Family Functioning (McMaster Model; Epstein, Bishop, & Levin, 1978; Epstein et al., 1983). A major advantage of the FAD is that it relates to a theoretical model that is used in clinical practice (Kabacoff, Miller, Bishop, Epstein, & Keitner, 1990). The McMaster Model is based on six dimensions: problem solving, communication, roles, affective responsiveness, affective involvement, and behavior control (Epstein et al., 1978; Miller, Ryan, Keitner, Bishop, & Epstein, 2000). These dimensions and an additional GF dimension make up the seven subscales of the FAD (Miller, Epstein, Bishop, & Keitner, 1985). It is these dimensions of the FAD that enable identification of healthy and unhealthy areas of functioning within a family (Byles et al., 1988; Staccini, Tomba, Grandi, & Keitner, 2015). Research supports the psychometric properties of the FAD as a measure of family functioning for use with nonclinical, psychiatric, and medical samples (Kabacoff et al., 1990; Sawyer, Sarris, Baghurst, Cross, & Kalucy, 1988; Staccini et al., 2015). As it captures multiple dimensions of family functioning, it has been used extensively across a wide range of populations to investigate a diverse set of issues (Miller et al., 2000). For research on the functioning of families the FAD is one of the most widely used assessment measures (Kabacoff et al., 1990; Mansfield, Keitner, & Dealy, 2015). The GF12 subscale of the FAD has also been validated as a single index for characterizing overall family functioning, with good psychometric properties (see Table 1 for items). It has high intercorrelations with the six dimensions of the FAD and with the principal components of the other 48 items (Kabacoff et al., 1990; Mansfield et al., 2015). The brevity and ease of administration of the GF12 has enabled several studies, including the Ontario Child Health Study (Byles et al., 1988) and the Western Australian Child Health Survey (Zubrick et al., 1995), to assess the health of families in large population-based samples. Several studies have suggested that the reliability of a measure is impacted by the format of questions and the mode of administration (Kalton & Schuman, 1982). The FAD and subsequently the GF12 subscale are designed as self-report measures which use positive and negatively worded items to capture both healthy and unhealthy family functioning (Byles et al., 1988; Epstein et al., 1983). However, the practice of negatively phrasing TABLE 1 General Functioning Subscale of the Family Assessment Device and Congeneric Factor Model of Positive Family Functiona. The 6 Positive Items are Indicated in Bold Type

Item Planning family activities is difficult because we misunderstand each other In times of crisis we can turn to each other for support We cannot talk to each other about the sadness we feel Individuals are accepted for what they are We avoid discussing our fears and concerns We can express feelings to each other There are lots of bad feelings in our family We feel accepted for what we are Making decisions is a problem for our family We are able to make decisions about how to solve problems We don’t get along well together We confide in each other

N = 3,225, SRMR = 0.03, RMSEA = 0.05.

a

Fam. Proc., Vol. 54, March, 2015

Item factor loading kx 0.699 0.722 0.785 0.802 0.730 0.844

118 /

FAMILY PROCESS

items to protect against acquiescence and response-set behaviors can introduce problems with internal consistency and factor structures (Barnette, 2000). One study conducted by Fristad (1989) which administered the full FAD resulted in a number of respondents needing clarification for several negatively worded questions. The purpose of this study was to examine the psychometric properties of a shortened version of the GF subscale of the FAD. Existing data from two previous studies were analyzed to assess the internal consistency and feasibility of using only the six positive items from the GF subscale (GF6+) as a measure of family functioning. Criterion validity of the GF6+ subscale was assessed by comparing it with the GF12 subscale and the Bradburn Scale (BS), a measure of psychological well-being, to determine whether associations observed between the BS and the GF12 subscale would be replicated using the short GF6+ version.

METHODOLOGY Participants The total number of participants used in this study was 3,225. Existing data from two previous studies (The Raine Study and the Western Australian Child Health Survey) were used. In both studies only the GF12 subscale of the FAD was administered. The Raine Study (RS): The Western Australian Pregnancy Cohort Study, also known as the RS, is a longitudinal health research project which started in 1989 and is still ongoing. It began with 2,900 pregnant women being recruited and assessed during their pregnancies. Data were collected on both parents, including information on work, health, and lifestyle. Information was also collected on the RS children regarding their physical and psychological development. In 1992, the GF12 subscale was completed as a self-report questionnaire by 1,959 primary caregivers, as part of the 3-year follow-up of the RS. Western Australian Child Health Survey (WACHS): The WACHS was a large scale epidemiological survey on the health and well-being of Western Australian children and their families. In 1993, face to face interviews, which contained the GF12 subscale, were conducted with 1,266 primary caregivers in the participants’ homes as part of the WACHS.

Measures General functioning subscale of the FAD As noted previously, the GF12 subscale is one of the dimensions of the FAD and has also been used as a single indicator to assess family functioning. The GF12 subscale is made up of 12 items, six items that reflect healthy family functioning and the other six items reflecting unhealthy functioning (Epstein et al., 1983). Scoring is on a 4-point scale (from 1 for strongly agree to 4 for strongly disagree) with the scale for the negatively worded items reversed. The total score is then divided by the number of items on the subscale giving a total score ranging from 1.0 (best functioning) to 4.0 (worse functioning) (Miller et al., 2000). The GF6+ consists of the six positive items from the GF subscale. Bradburn scale The BS consists of 10 items which are used as an indicator of psychological well-being (Bradburn, 1969). The two components of the scale measure positive and negative affect with five items for each component. The positive and negative affect components are scored separately. Response categories are “yes” or “no” and scores are derived from summing the yes responses in each scale (Harding, 1982; McDowell & Praught, 1982). High positive affect scores indicate high psychological well-being and high negative affect scores indicate low psychological well-being (Bradburn, 1969). www.FamilyProcess.org

BOTERHOVEN DE HAAN, HAFEKOST, LAWRENCE, SAWYER, & ZUBRICK

/ 119

Analyses The item pools in each of the RS and WACHS data were combined. After screening for missing data, outliers, and distributions, 3,225 cases were available for analyses. The general approach to analysis is predicated on the GF6+ subscale measuring a single factor. Therefore, we commenced our analyses with an examination of the univariate distributions of each of the six items and then fitted an ordinal confirmatory factor model. We selected the Standardized Root Mean Residual (SRMR ≤ 0.09) as our lead indicator of model fit. This was supplemented with the frequently used Root Mean Square Error of Approximation (RMSEA ≤ 0.06) (Hu & Bentler, 1999). In both the RS and WACHS data sets, we calculated the FAD-GF subscale items using the traditional scoring method for all 12 items, and also calculated a score from the six positively worded items by summing the item responses and dividing by six. We calculated correlations between the GF12 subscale score and the GF6+ subscale score, plotted the distribution of both sets of scores, and regressed the total score on the positive items score. We utilized the cut-off score of greater than 2.00 for categorical analyses, developed by Miller et al. (1985), which identifies healthy versus unhealthy families. They established this cut-off to satisfy both the theoretical and content perspectives of the FAD. We conducted a regression analysis between the GF12 and GF6+ subscale scores to determine if a score on one scale predicted an equivalent or higher or lower score on the other scale. This analysis suggested that the cut-off would be equivalent for the GF6+ subscale. We also undertook a sensitivity analysis to determine if a different cut-off on the GF6+ subscale would have higher concordance with classification on the GF12 subscale. The results of this analysis indicated that using the same cut-off scores of greater than 2.00 was optimal for determining unhealthy family functioning on both the GF12 and GF6+ subscales. Using the same cut-off score of greater than 2.00 we identified families that would be identified as having unhealthy functioning using both scales, either one or the other scale, or neither scale. To assess the criterion validity, we examined the relationship between family functioning and primary carer psychological well-being. The FAD has been validated for use with clinical populations, therefore, we postulated that there would be a relationship between carer reports of psychological distress and unhealthy family functioning. To assess this relationship we used the BS and both the GF12 and GF6+ subscales.

RESULTS An ordinal confirmatory factor model was used to fit items in the GF6+ subscale. This analysis indicated that a one-factor congeneric model fitted the data well (SRMR < 0.03; RMSEA < 0.05). Item loadings, fit indices, and scale reliability (H) (Hancock & Mueller, 2006) are reported in Table 1. The correlation between the GF12 subscale and the GF6+ was high. RS data showed a correlation of 0.909 (95% CI: 0.90–0.92). WACHS data showed a correlation of 0.890 (95% CI: 0.88–0.90). Table 2 below shows the distribution of the GF12 and the GF6+ subscale scores for families participating in the WACHS and the year 3 follow-up of the RS. Table 3 shows agreement between the GF12 subscale and the GF6+ subscale at the level of dichotomizing subscale scores into healthy and unhealthy family functioning. Although the correlation between the GF12 and the GF6+ subscale scores is high, there is some variability and respondents who are close to the cut-off point of 2.00 can be classified differently using either the GF12 or GF6+ item subscales. This can be seen from the lower average scores on both subscales for respondents who are unhealthy on one subscale only compared with those who are unhealthy on both subscales. Fam. Proc., Vol. 54, March, 2015

120 /

FAMILY PROCESS

TABLE 2 Distribution of the GF12 Subscale Scores and GF6+ Subscale Scores for Families Participating in Year 3 Follow-Up of the Raine Study and the 1993 WA Child Health Survey

GF6+ n 1993 WA Child Health Survey 1.0–1.1 175 1.1–1.3 95 1.3–1.5 91 1.5–1.7 209 1.7–1.9 184 1.9–2.1 301 2.1–2.3 95 2.3–2.5 44 2.5–2.7 50 2.7–2.9 7 2.9–4.0 15 Total 1,266 Year 3 follow-up of the Raine Study 1.0–1.1 329 1.1–1.3 204 1.3–1.5 195 1.5–1.7 446 1.7–1.9 251 1.9–2.1 276 2.1–2.3 94 2.3–2.5 69 2.5–2.7 51 2.7–2.9 17 2.9–4.0 27 Total 1,959

GF12 %

N

%

13.8 7.5 7.2 16.5 14.5 23.8 7.5 3.5 3.9 0.6 1.2 100.0

146 114 97 189 156 332 109 61 41 9 12 1,266

11.5 9.0 7.7 14.9 12.3 26.2 8.6 4.8 3.2 0.7 0.9 100.0

16.8 10.4 10.0 22.8 12.8 14.1 4.8 3.5 2.6 0.9 1.4 100.0

297 248 232 361 244 333 91 54 57 12 30 1,959

15.2 12.7 11.8 18.4 12.5 17.0 4.6 2.8 2.9 0.6 1.5 100.0

TABLE 3 Comparison of Classification of Families as Having Healthy or Unhealthy Functioning Using the GF12 and the GF6+ Subscales

Family functioning Raine study Unhealthy on both scales Unhealthy on total score only Unhealthy on positive items only Healthy on both scales Total 1993 WA child health survey Unhealthy on both scales Unhealthy on total score only Unhealthy on positive items only Healthy on both scales Total

Number of families

Mean GF12 total score

Mean GF6+ total score

187 57 71 1,644 1,959

2.53 2.28 1.97 1.49 1.63

2.54 1.82 2.22 1.49 1.62

162 70 49 985 1,266

2.43 2.26 2.01 1.59 1.75

2.44 1.90 2.21 1.57 1.72

As there were some cases that were classified differently between the GF12 and the GF6+ subscales, it was of interest to determine if this would have a meaningful impact on the analysis of data. Table 4 shows the relationship between family functioning and carer www.FamilyProcess.org

/ 121

BOTERHOVEN DE HAAN, HAFEKOST, LAWRENCE, SAWYER, & ZUBRICK

TABLE 4 Relationship between Family Functioning and Carer Psychological Well-being, Assessing Family Functioning Using the GF12 Subscale or the GF6+ Subscales

Family functioning

Carer psychological well-being—Bradburn Scale Raine study (n = 1,903) Lower third Middle third Upper third 1993 WA child health survey (n = 1,249) Lower third Middle third Upper third

Assessed using the GF12 subscale

Assessed using the GF6+ subscale

Healthy

Healthy

Unhealthy

Unhealthy

n

%

n

%

n

%

n

%

435 518 721

26 31 43

159 44 26

69 19 11

434 506 714

26 31 43

167 49 33

67 20 13

241 293 488

24 29 48

128 53 46

56 23 20

252 299 491

24 29 47

117 47 43

56 23 21

psychological well-being in both the RS and the WACHS. The BS scores were broken up into three approximately equal groups consistent with previous published reports from the WACHS. There is a clear and strong relationship between carer psychological wellbeing and unhealthy family functioning in both studies. Using the GF12 subscale, in the RS 69% of carers reported unhealthy family functioning if the carer was in the lower third of the BS compared with 11% reporting unhealthy family functioning when the carer was in the upper third of the BS. Using the GF6+ subscale produced corresponding percentages of 67% and 13%. A very similar association was seen in the WACHS. In both studies, use of the GF12 subscale or the GF6+ subscale produced virtually identical findings.

DISCUSSION The purpose of this study was to assess the reliability and validity of the GF6+ subscale, taken from the GF subscale of the FAD, as a measure of family functioning. Our analysis revealed that scores based on only the six positive items had almost identical distribution and identified almost exactly the same families with good and poor levels of functioning as the GF12 subscale. We found that the loss of precision using only the GF6+ subscale would be minor. Therefore, in consideration of mode and ease of administration, we conclude that the GF6+ subscale is a valid and useful measure to gain an overall perspective on family functioning. To assess the clinical utility of the GF6+ subscale, an analysis was conducted comparing scores from both the GF12 and the GF6+ subscales with primary carer psychological wellbeing using the BS. This analysis served as an indicator to establish what analytic discriminative power would be lost with the use of only six positive items for identifying unhealthy functioning families, with parents who reported poor psychological well-being. These findings demonstrated that overall the GF6+ subscale yielded virtually identical results as the GF12 subscale. As such, in situations where the GF12 subscale is considered to be a suitable measure of family functioning we suggest that the GF6+ subscale would be equally effective. Method of administration could be a potential explanation for the differences found in the average family functioning score between the WACHS and RS studies used in this Fam. Proc., Vol. 54, March, 2015

122 /

FAMILY PROCESS

analysis. The WACHS was a face to face interview in comparison to the RS which used a self-report questionnaire. The RS respondents reported better family functioning with higher average scores compared to the WACHS respondents. The FAD and subsequently the GF12 subscale were designed to be administered as self-report questionnaires. There are limitations arising from this research. The FAD was designed to be completed by family members aged 12 and over. However, in the studies we used for our analysis only the primary caregiver was administered the GF12 subscale. Discrepancies between different family members’ reports have been noted previously by Sawyer et al. (1988); therefore, we are restricted in generalizing our findings. In the WACHS and RS there was data missing for some of the families. Our analysis was restricted to families with no data missing for any of the GF12 subscale items. Additionally there were differences in the method of administration between these two studies. These methodological issues could potentially have created some bias within our sample. For future research, further analysis would need to be conducted to identify if the shortened six positive item version of the GF subscale adequately encapsulates each of the dimensions of the FAD. Additionally replication studies are required to further investigate the reliability and validity of the GF6+ subscale. It should be noted that the GF6+ subscale should not be used to replace a full clinical assessment to identify areas of unhealthy functioning in families who present for treatment. With that in mind our study is the first to suggest that the GF6+ subscale has potential as a tool to measure family functioning in large-scale studies where contextual information is desired. A short item scale which is easy to administer and reduces respondent burden would provide greater opportunity for capturing an overall picture of the functioning of families at population levels. REFERENCES Barnette, J. J. (2000). Effects of stem and Likert response option reversals on survey internal consistency: If you feel the need, there is a better alternative to using those negatively worded stems. Educational and Psychological Measurement, 60(3), 361–370. doi:10.1177/00131640021970592. Bradburn, N. M. (1969). The structure of psychological well-being. Chicago, IL: Aldine. Byles, J., Byrne, C., Boyle, M. H., & Offord, D. R. (1988). Ontario Child Health Study: Reliability and validity of the general functioning subscale of the McMaster Family Assessment Device. Family Process, 27(1), 97–104. doi:10.1111/j.1545-5300.1988.00097.x. Epstein, N. B., Baldwin, L. M., & Bishop, D. S. (1983). The McMaster family assessment device. Journal of Marital and Family Therapy, 9(2), 171–180. doi:10.1111/j.1752-0606.1983.tb01497.x. Epstein, N. B., Bishop, D. S., & Levin, S. (1978). The McMaster Model of family functioning. Journal of Marital and Family Therapy, 4(4), 19–31. doi:10.1111/j.1752-0606.1978.tb00537.x. Fristad, M. A. (1989). A comparison of the McMaster and Circumplex family assessment instruments. Journal of Marital and Family Therapy, 15(3), 259–269. doi:10.1111/j.1752-0606.1989.tb00808.x. Hancock, G. R., & Mueller, R. O. (2006). Structural equation modeling: A second course. Greenwich, CT: Information Age Publishing. Harding, S. D. (1982). Psychological well-being in Great Britain: An evaluation of the Bradburn Affect Balance Scale. Personality and Individual Differences, 3(2), 167–175. doi:10.1016/0191-8869(82)90031-9. Hu, L., & Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling: A Multidisciplinary Journal, 6(1), 1–55. doi:10. 1080/10705519909540118. Kabacoff, R. I., Miller, I. W., Bishop, D. S., Epstein, N. B., & Keitner, G. I. (1990). A psychometric study of the McMaster Family Assessment Device in psychiatric, medical, and nonclinical samples. Journal of Family Psychology. Series A (General), 3(4), 431. doi:10.1037/h0080547. Kalton, G., & Schuman, H. (1982). The effect of the question on survey responses: A review. Journal of the Royal Statistical Society, 145(1), 42–73. doi:10.2307/2981421. Kessler, R. C., Andrews, G., Colpe, L. J., Hiripi, E., Mroczek, D. K., Normand, S. L. et al. (2002). Short screening scales to monitor population prevalences and trends in non-specific psychological distress. Psychological Medicine, 32(6), 959–976. doi:10.1017/S0033291702006074.

www.FamilyProcess.org

BOTERHOVEN DE HAAN, HAFEKOST, LAWRENCE, SAWYER, & ZUBRICK

/ 123

Kessler, R. C., Barker, P. R., Colpe, L. J., Epstein, J. F., Gfroerer, J. C., Hiripi, E. et al. (2003). Screening for serious mental illness in the general population. Archives of General Psychiatry, 60(2), 184–189. doi:10.1001/ archpsyc.60.2.184. Mansfield, A. K., Keitner, G. I., & Dealy, J. (2015). The family assessment device: An update. Family Process, 54(1), 82–93. doi:10.1111/famp.12080. McDowell, I., & Praught, E. (1982). On the measurement of happiness: An examination of the Bradburn Scale in the Canada Health Survey. American Journal of Epidemiology, 116(6), 949–958. Miller, I. W., Epstein, N. B., Bishop, D. S., & Keitner, G. I. (1985). The McMaster family assessment device: Reliability and validity. Journal of Marital and Family Therapy, 11(4), 345–356. doi:10.1111/j.1752-0606.1985. tb00028.x. Miller, I. W., Ryan, C. E., Keitner, G. I., Bishop, D. S., & Epstein, N. B. (2000). The McMaster approach to families: Theory, assessment, treatment and research. Journal of Family Therapy, 22(2), 168–189. doi:10.1111/ 1467-6427.00145. Sawyer, M. G., Sarris, A., Baghurst, P. A., Cross, D. G., & Kalucy, R. S. (1988). Family Assessment Device: Reports from mothers, fathers, and adolescents in community and clinic families. Journal of Marital and Family Therapy, 14(3), 287–296. doi:10.1111/j.1752-0606.1988.tb00748.x. Sharp, L. M., & Frankel, J. (1983). Respondent burden: A test of some common assumptions. Public Opinion Quarterly, 47(1), 36–53. doi:10.1086/268765. Staccini, L., Tomba, E., Grandi, S., & Keitner, G. I. (2015). The evaluation of family functioning by the family assessment device: A systematic review of studies in adult clinical populations. Family Process, 54(1), 94–115. doi:10.1111/famp.12098. Zubrick, S. R., Silburn, S., Garton, A., Burton, P., Dalby, R., Carlton, J. et al. (1995). Western Australian Child Health Survey: Developing health and well-being in the nineties. Perth, WA: Australian Bureau of Statistics and the Institute for Child Health Research.

Fam. Proc., Vol. 54, March, 2015

Reliability and validity of a short version of the general functioning subscale of the McMaster Family Assessment Device.

The General Functioning 12-item subscale (GF12) of The McMaster Family Assessment Device (FAD) has been validated as a single index measure to assess ...
85KB Sizes 21 Downloads 8 Views