Journal of Marital and Family Therapy doi: 10.1111/jmft.12095 October 2014, Vol. 40, No. 4, 525–534
THE DEVELOPMENT OF A RELIABLE CHANGE INDEX AND CUTOFF FOR THE REVISED DYADIC ADJUSTMENT SCALE Shayne R. Anderson, Rachel B. Tambling, Scott C. Huff, and Joy Heafner University of Connecticut
Lee N. Johnson Brigham Young University
Scott A. Ketring Auburn University
The Revised Dyadic Adjustment Scale (RDAS; Busby, Crane, Larson, & Christensen, 1995) is a measure of couple relationship adjustment that is often used to differentiate between distressed and non-distressed couples. While the measure currently allows for a determination of whether group mean scores change significantly across administrations, it lacks the ability to determine whether an individual’s change in dyadic adjustment is clinically significant. This study addresses this limitation by establishing a cutoff of 47.31 and reliable change index of 11.58 for the RDAS by pooling data across multiple community and clinical samples. An individual whose score on the RDAS moves across the cutoff changes by 12 or more points can be classified as experiencing clinically significant change. The effectiveness of therapy can be evaluated in several ways. Howard, Moras, Brill, Martinovich, and Lutz (1996) identified three different types of outcome research, each of which answers different questions about the effectiveness and efficacy of therapy. First, efficacy research answers the question “In general, does this treatment work under controlled laboratory settings?” This can be contrasted with effectiveness research, which answers the question, “In general, does this treatment work under normal clinical conditions?” While both of these questions are important to the field of family therapy, both deal with change at the group level by examining change in the treatment group’s mean scores across time. Neither answers the question of fundamental importance to clinicians—“Is the treatment I am providing helping this particular couple or family?” This third type of more individualized outcome research has direct application for practitioners. It allows clinicians to monitor treatment as it is provided to clients and to accurately assess the extent to which the treatment is influencing client progress on outcomes of interest. By monitoring the provision of treatment and client progress, clinicians obtain evidence that can be used to demonstrate the effectiveness of their work. To track client progress, clinicians need measures that can assess whether clinically significant change is occurring on the constructs of interest (Johnson & Miller, 2014). Of critical importance to family therapists is the question of whether or not a treatment is effective in improving the quality of couple or family relationships. In an attempt to accurately assess relationship quality, a number of self-report measures of dyadic adjustment and satisfaction have emerged including the Dyadic Adjustment Scale (Spanier, 1976), Locke Wallace Marital Adjustment Test (Locke & Wallace, 1959), and the Couple Satisfaction Index (CSI; Funk & Rogge, 2007). One commonly
Shayne R. Anderson, PhD, Rachel B. Tambling, PhD, Scott C Huff and Joy Heafner, Department of Human Development and Family Studies, University of Connecticut; Lee N. Johnson, PhD, Marriage and Family Therapy Program in the School of Family Life, Brigham Young University; Scott. A Ketring, PhD, Department of Human Development and Family Studies, Auburn University. Address correspondence to Shayne R. Anderson, Department Human Development and Family Studies, Unit 1058, University of Connecticut, Storrs, Connecticut 06269-1058; E-mail: [email protected]
October 2014
JOURNAL OF MARITAL AND FAMILY THERAPY
525
used measure of relationship adjustment, the Revised Dyadic Adjustment Scale (RDAS; Busby, Crane, Larson, & Christensen, 1995), has enjoyed both popularity and empirical support, with translations available in French, Portuguese, and Spanish (Hollist et al., 2012; Mead, Thurber & Crane, 2003; Vandeleur, Fenton, Ferrero & Presig, 2003). The RDAS consists of fourteen items extracted from the Dyadic Adjustment Scale (DAS; Spanier, 1976). One of the strengths of the RDAS is that it is able to assess three aspects of dyadic adjustment (consensus, satisfaction, and cohesion) in a valid and reliable way with only 14 items. While similar length scales such as the CSI-16 measure satisfaction more reliably than the RDAS, they are limited to a focus on satisfaction. Researchers and clinicians wishing to assess couple relationships more broadly turn to the RDAS because it has been used in numerous studies with good reliability and is highly correlated with the DAS. Previous research also indicates that the RDAS is successfully able to distinguish between distressed and non-distressed relationships (Busby et al., 1995). Despite the relative strength of the RDAS in comparison with similar instruments, we do not know whether the RDAS can accurately assess whether an individual’s experience of the relationship has changed in a meaningful way as a result of treatment. The purpose of this study is to provide clinicians and researchers with the information to identify whether clients have made meaningful changes in their dyadic adjustment as measured by the RDAS.
ASSESSING CLINICALLY SIGNIFICANT CHANGE Most research in couple therapy uses statistical significance to evaluate whether an intervention is associated with client improvement. While statistical significance has an important role in research, it cannot assess whether differences that occur as a result of treatment are meaningful. It is well documented that given a large enough sample, any difference, regardless of its real world importance can be statistically significant (Cohen, 1994). To assess whether changes are meaningful, we need to move beyond statistical significance to clinical significance. Despite the practical importance of assessing clinical significance, it has not gained sufficient traction in the field of couple and family therapy. While the clinical significance of couple therapy was discussed in the 1990s (Bray & Jouriles, 1995), a similar focus has not been reflected in recent reviews of couple therapy (Lebow, Chambers, Christensen, & Johnson, 2012) or marital enrichment (Markman & Rhoades, 2012), nor was a measure’s ability to assess clinical significance listed as a criteria in evaluating the methodological strength of research in ten substantive areas of family therapy (Sprenkle, 2012). While there are multiple procedures for determining clinical significance, the most widely used is the Jacobson and Truax (1991) method. This method requires that two conditions be met to identify change as clinically significant. First, an individual’s score from pre- to post-test must pass from the dysfunctional to the functional population, typically across a pre-determined cutoff score. Second, the change must be statistically reliable (i.e., large enough to not be due to chance), as measured by a reliable change index (RCI). There have been many attempts to improve upon the Jacobson and Truax method by using various estimate interval approaches. These approaches result in few differences in classification of clinical significance (Atkins, Bedics, McGlinchey, & Beauchaine, 2005) leading to the conclusion that that the classic approach is preferable (Maassen, 2000). The Jacobson and Truax method requires both a cutoff and a RCI to determine whether clients are making clinically significant progress. The RDAS, which lacks an empirically derived cutoff and RCI, cannot currently be used to determine whether clinically significant change has taken place.
THE DEVELOPMENT OF A CUTOFF SCORE ON THE RDAS Crane, Middleton, and Bean (2000) have recommended a cutoff score of 48 for the RDAS. While the cutoff score Crane and associates developed has been useful in helping clinicians and researchers distinguish between distressed and non-distressed couples, it was derived from a regression equation that converts the cutoff on the well-established DAS to an equivalent score on the RDAS. There are two limitations to this approach. First, Crane and associates used a liberal cutoff of 107 on the original Dyadic Adjustment Scale to discriminate between distress and adjustment in their regression equation. This cutoff is typically used to identify well-adjusted individuals. Cutoffs to identify distressed individuals are substantially lower (100: Spanier, 1976; 92: 95: Sabourin, 526
JOURNAL OF MARITAL AND FAMILY THERAPY
October 2014
Valois, & Lussier, 2005). Second, unless the regression equation is perfectly predictive, measurement error can bias the identified cutoff score. A preferable approach is to identify the means of a healthy and distressed population on the measure in question. Jacobson and Truax (1991) suggest the following equation for determining a cutoff score based on two unequal distributions where s represents the standard deviation, M the mean, and 0 or 1 the community and clinical samples: ð1Þ
c¼
s0 M1 þ s1 M0 s0 þ s1
Such a method of determining the cutoff score allows researchers and clinicians to make a determination about which population a given score falls within. Thus, change scores based on cutoff scores that use this method will account for measurement error and unique characteristics of the measure in question.
THE DEVELOPMENT OF A CRITERION FOR CLINICALLY SIGNIFICANT CHANGE ON THE RDAS The second criterion in the Jacobson and Truax (1991) method of establishing clinically significant change is determining that the change is statistically reliable. Classical test theory assumes that an observed score on a measure like the RDAS is a combination of the true score (i.e., actual dyadic adjustment) and measurement error (i.e., the portion of the score that is due to factors not associated with one’s dyadic adjustment). To increase our confidence that changes in scores on the RDAS across administrations represent real changes in an individual’s actual dyadic adjustment, a reliable change index (RCI) must be established. The RCI represents the level of change necessary to be confident that the difference in scores across administrations is not due to chance or error but rather reflects an actual change. To determine a reliable change index, Jacobson and Truax (1991) have suggested the following equation: ð2Þ
RCI ¼
x2 x1 Sdiff
Where x2x1 represents an individual’s change between administrations of the instrument. Sdiff the standard error (SE) of the difference between the two scores, is defined in the following equations: qffiffiffiffiffiffiffiffiffiffiffiffiffiffi ð3Þ Sdiff ¼ 2ðSE Þ2 ð4Þ
SE ¼ s1
pffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi 1 rxx
Sdiff accounts for the variation in reliability of the measure. It represents the standard deviation of the clinical population at intake (s1) and the test–retest reliability (rxx) of the measure in a non-clinical sample. In order for a change to be deemed statistically reliable at the p = .05 level, it must be approximately twice (1.96) the standard error of the difference between the scores. If the change exceeds this threshold, researchers and clinicians can have increased confidence that a difference in scores between administrations of a measure are due to an actual change in the construct of interest rather than error. The goal of this study is to provide clinicians and researchers with the information they need to use the RDAS to determine whether the changes in dyadic adjustment that their clients or research participants are reporting are reliable and clinically significant. In particular, we will answer two questions: 1. What is the cutoff that best distinguishes a clinical population from a community population on the RDAS? 2. How much does an individual’s score on the RDAS need to change to be statistically reliable (i.e., what is the reliable change index for the RDAS)?
October 2014
JOURNAL OF MARITAL AND FAMILY THERAPY
527
METHOD Sample In the absence of normative data, Jacobson and Truax (1991) recommend pooling data across studies that use the RDAS in clinical and community populations. Each study sample is viewed as one of many possible samples drawn from the same distressed or non-distressed population. Pooling data allows for increased heterogeneity of samples and reduces the potential bias using estimates from any given sample could introduce. The data that are pooled in this study come from three sources: Data from community and clinical administrations of the RDAS published in the literature, additional clinical data from three university-based marriage and family therapy training clinics, and a community sample recruited to establish the test–retest reliability of the RDAS. These pooled samples represent 4967 individuals (60% female) in the community samples and 1228 individuals (48% female) in the clinical samples. Table 1 provides the sources of community and clinical samples used in this study. Published studies that have used the RDAS. To identify studies for pooling, we reviewed all studies in the PsychInfo database that cited the article in which the RDAS was developed (Busby et al., 1995). Studies were included if they met the following criteria: (a) The study was peerreviewed research that used the complete RDAS; (b) the study provided sample size, mean, and standard deviation; (c) the study used the English version of the measure; (d) the data in the study were not reported in another study or dataset included in the review; and (e) participants were receiving couple treatment for relational distress or were a part of a community sample. Of the 158 peer-reviewed citations of the RDAS, twenty-two studies met the inclusion criteria. Six studies
Table 1 Sources of Community and Clinical Data Used in Pooled Estimates Community samples
Clinical samples
Blow et al. (2013) Bradford (2012)
Anderson & Miller (2006) Auburn University. MFT Training Clinic Database Busby, Crane, Christensen & Larsen (1995) DeMaria (2005) Gangamma,Bartle-Haring & Glebova (2012)
Bridgett, Burt, Laake & Oddi (2013) Busby, Crane, Christensen, & Larsen (1995) Christie, Meyerowitz, Stanton, Rowland & Ganz (2013) Cook & Jones (2002) Frye-Cox & Hesse (2013) Futris, Campbell, Nielsen & Burwell (2010) Hawkins, Fawcett, Carroll & Gilliland (2006) Hurley, Field, T. & Bendell-Estoff (2012)
McLean, Walton, Rodin, Esplen, & Jones (2013) Olson & Russell (2004) Schweinle & Ickes (2007) University of Connecticut MFT Training Clinic Database University of Georgia MFT Training Clinic Database
Kim, Johnson & Ripley (2011) Miller (2000) Nelson, O’Brien, Blankson, Calkins & Keane (2009) Ott, Sanders & Kelber (2007) Schramm, Marshall, Harris & Lee (2005) Test-Retest Reliability Sample Yorgason, Almeida, Neupert & Hoffman (2006)
528
JOURNAL OF MARITAL AND FAMILY THERAPY
October 2014
provided only clinical data, 15 provided only community data, and one included both a clinical and community sample. Over half of the existing studies (59%) used in this study collected data from both partners of the couple. To account for this non-independence, either the male or female data from each study was chosen at random to include in the pooling. Data were re-analyzed with the other partner’s data to check for bias. Results indicate that choice of partner did not lead to any meaningful changes in the pooled mean or standard deviation. The community and clinical samples were both nearly equally divided between men and women, although the community samples skewed towards more female (63% on average) and the clinical samples skewed towards male (52% on average). The community samples were older (average mean age was 43) than the clinical samples (average mean age was 34). Both samples represented couples from across the United States and were predominately White. Rates of marriage were higher in the community samples (90%) than the clinical samples (77%), although the community samples’ average length of time together was only 2 years longer. Education levels were comparable between the samples, but the community sample reported higher incomes on average. Online supplemental tables provide demographic information, reliability estimates, means and standard deviations for all samples that were pooled in this study. These supplemental tables are included next to the link to this article on the JMFT website. Additional clinical samples. Participants were selected from the archived data of individuals who received couple therapy at one of three family therapy training clinics, located in the Northeast and the Southeast regions of the United States. Individuals, couples, and families with a wide range of presenting problems were treated in the university training clinics by masters or doctoral students enrolled in programs accredited by the Commission on the Accreditation for Marriage and Family Therapy Education. Faculty members in each program provided clinical supervision. Clients were normally seen on a weekly basis and charged on a sliding fee scale based on family income and the number of people supported by that income. Discounted treatment was offered to university students. Clients who consented to participate in research allowed their responses in the clinical assessment packets to be used for research purposes. Prior to the first session, clients completed an intake assessment packet. In addition to demographic information, the client intake assessment packet included several self-report measures of distress, psychological adjustment, and couple relationship functioning including the RDAS. It was important in this study for the clinical sample to represent the population of interest: couples experiencing dyadic distress who are working on ameliorating that distress. After identifying couples receiving therapy, we reviewed the stated presenting problems and removed from the sample those couples who were clearly seeking premarital therapy or who were separated/divorced and working on co-parenting issues. Due to the significant interdependence of couple data, we chose one member from each dyad. If only one partner had complete data on the RDAS, that partner was chosen. When both members of the dyad had complete data we chose the male partner to obtain a more balanced sample. Participants (n = 97) at the Northeastern clinic had a mean age of 31.31 (SD = 10.27). Approximately half were married (48.5%) with an average length of relationship of 4.95 years (SD = 4.21). There were slightly more females than males (54.6%). The sample was predominantly White (73.2%) and well educated (44.2% bachelor’s degree or more), with an average salary range of $40,000 or higher. The mean RDAS score of this sample was 41.74 (11.57). Participants at the first Southeastern clinic (n = 229) were predominantly female (65.9%) and married (57.3%), with an average relationship length of 5.13 years (SD = 5.74). The sample was generally young (M = 30.92, SD = 7.83), White (85%), well educated (62.1% bachelor’s degree or more), and reported a median income range of $20–29,000. The mean RDAS score of this sample was 41.26 (SD = 10.67). Similar to clients at the first two clinics, clients at the second Southeastern clinic (n = 330) reported a mean age of 31.19 (SD = 8.29), were White (76%) and were well-educated, with half reporting receiving at least a bachelor’s degree. This sample was equally divided between men and women (50.6% female) with an average salary range of $30–39,000. Most were married (82.1%)
October 2014
JOURNAL OF MARITAL AND FAMILY THERAPY
529
with an average length of relationship of 7.03 years (SD = 6.06). The mean RDAS score of this sample was 38.39 (SD = 9.92). Test–retest reliability sample. Only one previous study has examined the test–retest reliability of the RDAS. Hawkins, Fawcett, Carroll, and Gilliland (2006) reported 8-week reliability estimates of 0.78 for men and 0.68 for women. These figures likely underestimate the reliability of the RDAS as they were generated during the second and third trimesters from a sample of parents expecting their first child. It is likely that some change in dyadic adjustment occurred during this critical 8-week period, thus lowering the test–retest reliability estimate. To get a more accurate estimate of the true test–retest reliability for the general population, an additional sample was drawn from a larger study examining the measurement properties of several clinical instruments. Participants for the larger study were recruited through advertisements posted to university bulletin boards, online classifieds sites (i.e., the top ten markets on craigslist.org and nationwide on backpage.com), and university student and faculty daily e-newsletters. Participants who completed the study were also given the option to invite their friends to participate through their social media accounts or via e-mail. Of the 1001 people to view the information page at the beginning of the survey, 365 consented to the project and met the eligibility criteria. Of these, 266 completed the first survey. One hundred ninety participants indicated that they were in a committed relationship and completed the RDAS during their first survey. Participants left an e-mail address at the end of the survey that researchers used to contact participants at two, three, and 4 weeks following their first survey. A unique identifier was used to link the first and second surveys while maintaining anonymity. Of the 190 participants who completed the RDAS during the first survey, 88 completed the second survey and provided the unique identifier that matched their first survey. The test–retest period chosen for this study was 2–4 weeks. Excluding participants who completed the survey outside of this time frame resulted in a final sample of 70 participants. Participants were predominantly female (78.6%) White (62.7%) and young, with 66% reporting an age range of 18–29. Most were not married (37.1%) but in a significant committed relationship with a median range of 3–4 years duration. The sample was well educated with approximately 61% reporting having received at least a bachelor’s degree. The median income range of this sample was $40–49,000. The mean RDAS score of this sample was 51.23 (SD = 7.58) at the first administration and 51.24 (SD = 7.81) at the second administration. Measures Revised Dyadic Adjustment Scale. The Revised Dyadic Adjustment Scale (RDAS, Busby et al., 1995) is a shortened version of the Dyadic Adjustment Scale (Spanier, 1976). The RDAS consists of fourteen items designed to measure adjustment in dyadic relationships on three subscales: Consensus, Satisfaction, and Cohesion. The subscales can be summed to create a total score representative of overall dyadic adjustment with a range of 0–69. The total score is used in this study. The RDAS correlates highly with the original DAS (r = .97; Busby et al., 1995) as well as similar measures of dyadic distress (Crane et al., 2000). Busby et al. (1995) factor analysis of the RDAS, also provided support for the construct validity of the measure. In past research, the RDAS has demonstrated strong reliability, with split-half reliabilities ranging from 0.90 to 0.95 for the total scale (Busby et al., 1995). Internal consistency estimates across studies that have used the RDAS have been consistently lower with a mean Chronbach’s alpha of .85 across the samples that provided reliability estimates in this study (range = .76–.91). Cronbach’s alphas from the three university clinics were .86, .87, and .89 while the alpha of the test–retest sample was .80.
RESULTS The 14–28 day test–retest reliability of the test–retest sample was .82. The pooled means and standard deviations for the clinical and community samples on the RDAS are presented in Table 2. Using this information in Jacobson and Truax’s (1991) formula (Equation 1) resulted in a cutoff score of 47.31. Jacobson and Truax’s (1991) method for determining reliable change was used to determine the amount of change in score from pretest to posttest that would be statistically significant at 530
JOURNAL OF MARITAL AND FAMILY THERAPY
October 2014
the p = .05 level. We used the community sample’s test–retest reliability estimate (.82), as well as the standard deviation of the clinical sample at intake presented in Table 2 as inputs for Equations 3 and 4. This resulted in an Sdiff for our sample of 5.91. Using this value in Equation 2 to solve for the level of change that would yield a significant value (i.e., 1.96) resulted in a Reliable Change Index of 11.58.
DISCUSSION This study establishes a cutoff and reliable change index for the RDAS using pooled estimates of the means and standard deviations across multiple community and clinical samples. An individual whose score on the RDAS moves across the cutoff to a score of 48 and changes by 12 or more points from the first to the most recent administration can be classified as experiencing clinically significant change. Changes of at least 12 points that do not cross the cutoff can be classified as reliable improvement or reliable deterioration based on the direction of change. Both the cutoff and the reliable change index can be helpful to clinicians who are attempting to operate a practice in accordance with results-based accountability standards. Clinicians can use these values to measure the effectiveness of treatment for each particular client couple. In addition, these values can assist researchers in establishing the efficacy of various treatments, and in describing clinical change. While it has been important to establish the values, the results of this research suggest that there is room to improve the precision with which the RDAS measures change in dyadic adjustment across time. Of primary concern is the value of the RCI relative to the range of the measure. The RDAS has a range of 70 points. A reliable change index for this measure represents a change of 16.6% of the measure’s range. In contrast, the 16-item version of the Couple Satisfaction Index (Funk & Rogge, 2007), a similar measure to the RDAS but focused solely on satisfaction, has a range of 82 points and an RCI of 9.4 (or 11.5% of the measure’s range; RD Rogge, JL Funk, S Lee & MC Saavedra, unpublished data). Other scales such as the Outcome Questionnaire-45.2 (OQ-45.2; Lambert et al., 1996), a popular measure for examining client progress in individual therapy, have even more narrow bands of reliable change. The OQ-45.2 has a range of 181 points and an RCI of 14 (or 7.8% of the measure’s range). Proportionately, an individual’s change score on the RDAS would have to be 2.1 times greater than the change score on the OQ-45.2 to register as true change. It is likely that the discrepancy in the RCI of these various measures is, in part, an artifact of the length of the measure. Reliability of a measure generally increases as the number of items increases, with increased reliability translating directly into a lower RCI. It is also possible, however, that the lower RCI of these measures comes from the degree of measurement refinement involved in their creation. For example, the CSI was developed using item-response theory, an alternative to classical test theory that results in more reliable measures. The OQ-45.2 was developed specifically as a measure of change in general psychological distress, leading the developers to focus on sensitivity to change throughout its development (Lambert et al., 1996). The reliable change index results in a ceiling for those that can experience reliable change. For example, individuals whose pretreatment RDAS score is 58 cannot reliably improve. While this may not be an issue for clinically distressed relationships, those wishing to use the RDAS to measure improvement at the individual level with non-distressed populations, such as those that present for premarital treatment, may have to search elsewhere for a measure to capture change in
Table 2 Pooled Estimates of Clinical and Community Samples, Cutoff, and RCI of the RDAS
RDAS
October 2014
Couple Therapy Sample
Community Sample
Mp
SDp
n
Mp
SDp
n
Cutoff
RCI
40.58
9.84
1,228
52.95
8.26
4,967
47.31
11.58
JOURNAL OF MARITAL AND FAMILY THERAPY
531
couples’ adjustment. It is possible that additional measurement refinement can make the RDAS a more useful tool for clinicians and researchers wishing to measure change in dyadic adjustment. Such refinement could include improving or replacing items that are particularly unstable across short time periods in community samples. By increasing the short-term temporal stability of the measure in community samples, the test-retest reliability, and consequently the RCI of the measure could be improved. For example, if the reliability were improved from .82 to .92, the RCI would drop almost four points to 7.7. The cutoff score of 47.31 identified in this study is less than one point lower than the score of 48 suggested by Crane et al. (2000). Despite the critique of the original RDAS cutoff, which converted the cutoff on the Dyadic Adjustment Scale to the RDAS using a regression equation, the cutoff proposed in this manuscript is functionally equivalent to the original. The lowest possible non-distressed score is 48 using either cutoff. While the results of this study only alter the cutoff minimally, by using the pooled means and standard deviations across multiple community and clinical samples, we can have more confidence that the cutoff accurately assesses the point at which an individual’s score on the RDAS passes from a clinical to a community population. Limitations The optimal community sample to use for establishing a cutoff score is a normative sample drawn from a national study. In the absence of a normative sample, we have relied on pooled data from multiple clinical and community samples. The replication of samples that are then pooled can provide a better estimate of the true population mean and standard deviation than any one sample alone. However, if a common selection effect occurred across samples, the resulting pooled mean may exhibit bias. This is of most concern in the clinical samples, the majority of which were drawn from university-based family therapy training clinics. It is possible that couples presenting for therapy at training clinics differ in significant ways from those presenting for treatment in other settings. This limitation should be taken into consideration when generalizing these results to broader clinical settings. While a predominantly university-clinic sample does impact the generalizability of the results to settings outside university clinics, it also makes the results particularly applicable to other university clinics where clinical research on couple interventions is often conducted. The sample used to establish the test–retest reliability estimate and the interval between administrations should also be taken into consideration when using the proposed RCI. The RCI identified in this study relies on a 2–4 week test–retest reliability estimate from a convenience sample. While convenience samples are the norm for establishing the test–retest reliability of an instrument, it is possible that the estimate would differ if a representative sample were used. The 2–4 week interval between administrations was chosen to balance recollection bias and the possibility that true change had occurred. Altering this interval would likely alter the test–retest reliability of the measure with shorter intervals increasing the estimate and longer intervals decreasing the estimate. Any changes to the test–retest reliability estimate would have an effect on the resulting RCI. Despite these limitations and the importance of further refining the RDAS suggested earlier, this study has offered clinicians who desire to measure the effectiveness of their clinical work with each client a way to do so. Using the improved cutoff and RCI will also allow researchers who use the RDAS to more accurately gauge whether the interventions they are testing lead to clinically meaningful changes in dyadic adjustment. It is our hope that clinicians and researchers will use this information to document the effectiveness of their clinical practice to provide consumers of therapy with evidence of the effectiveness of a given treatment. Furthermore, we hope that clinicians will use this information to monitor the progress of couples’ dyadic adjustment in therapy, allowing them to modify treatment plans when their client’s relational adjustment is not responding to treatment. We also hope that researchers will expand measurement work beyond initial validity and reliability testing to provide cutoffs, reliable change indices, and other information that will enhance the field’s ability to use formal assessment in clinical practice and clinical research. As researchers and clinicians examine the clinical significance of the work that they do, we are confident that both clients and the field of MFT will benefit.
532
JOURNAL OF MARITAL AND FAMILY THERAPY
October 2014
REFERENCES Anderson, S. R., & Miller, R. B. (2006). The effectiveness of therapy with couples reporting a history of childhood sexual abuse: An exploratory study. Contemporary Family Therapy, 28, 353–366. Atkins, D. C., Bedics, J. D., McGlinchey, J. B., & Beauchaine, T. P. (2005). Assessing clinical significance: Does it matter which method we use? Journal of Consulting and Clinical Psychology, 73, 982–989. Blow, A. J., Gorman, L., Ganoczy, D., Kees, M., Kashy, D. A., Valenstein, M., et al. (2013). Hazardous drinking and family functioning in National Guard veterans and spouses postdeployment. Journal of Family Psychology, 27, 303–313. Bradford, K. (2012). Assessing readiness for couple therapy: The Stages of Relationship Change Questionnaire. Journal of Marital and Family Therapy, 38, 486–501. Bray, J. H., & Jouriles, E. N. (1995). Treatment of marital conflict and prevention of divorce. Journal of Marital and Family Therapy, 21, 461–473. Bridgett, D. J., Burt, N. M., Laake, L. M., & Oddi, K. B. (2013). Maternal self-regulation, relationship adjustment, and home chaos: Contributions to infant negative emotionality. Infant Behavior and Development, 36, 534–547. Busby, D. M., Crane, D. R., Larson, J. H., & Christensen, C. (1995). A revision of the Dyadic Adjustment Scale for use with distressed and nondistressed couples: Construct hierarchy and multidimensional scales. Journal of Marital and Family Therapy, 21, 289–308. Christie, K. M., Meyerowitz, B. E., Stanton, A. L., Rowland, J. H., & Ganz, P. A. (2013). Characteristics of breast cancer survivors that predict partners’ participation in research. Annals of Behavioral Medicine, 46, 107–113. Cohen, J. (1994). The earth is round (p < .05). American Psychologist, 49, 997–1003. Cook, J. L., & Jones, R. M. (2002). Congruency of identity style in married couples. Journal of Family Issues, 23, 912–926. Crane, D. R., Middleton, K. C., & Bean, R. A. (2000). Establishing criterion scores for the Kansas Marital Satisfaction Scale and the Revised Dyadic Adjustment Scale. American Journal of Family Therapy, 28, 53–60. DeMaria, R. M. (2005). Distressed Couples and Marriage Education. Family Relations: An Interdisciplinary Journal of Applied Family Studies, 54, 242–253. Frye-Cox, N. E., & Hesse, C. R. (2013). Alexithymia and marital quality: The mediating roles of loneliness and intimate communication. Journal of Family Psychology, 27, 203–211. Funk, J. L., & Rogge, R. D. (2007). Testing the ruler with item response theory: Increasing precision of measurement for relationship satisfaction with the Couples Satisfaction Index. Journal of Family Psychology, 21, 472–583. Futris, T. G., Campbell, K., Nielsen, R. B., & Burwell, S. R. (2010). The Communication Patterns Questionnaire Short Form: A review and assessment. The Family Journal, 18, 275–287. Gangamma, R., Bartle-Haring, S., & Glebova, T. (2012). A study of contextual therapy theory’s relational ethics in couples in therapy. Family Relations: An Interdisciplinary Journal of Applied Family Studies, 61, 825–835. Hawkins, A. J., Fawcett, E. B., Carroll, J. S., & Gilliland, T. T. (2006). The marriage moments program for couples transitioning to parenthood: Divergent conclusions from formative and outcome evaluation data. Journal of Family Psychology, 20, 561–570. Hollist, C. S., Falceto, O. G., Ferreira, L. M., Miller, R. B., Springer, P. R., Fernandes, C. C., et al. (2012). Portuguese translation and validation of the revised Dyadic Adjustment Scale. Journal of Marital and Family Therapy, 38, 348–358. Howard, K. I., Moras, K., Brill, P. L., Martinovich, Z., & Lutz, W. (1996). Evaluation of psychotherapy: Efficacy, effectiveness, and patient progress. American Psychologist, 51, 1059–1064. Hurley, E. C., Field, T., & Bendell-Estoff, D. (2012). Rejection sensitivity and marital adjustment among military spouses during deployments. Psychology, 3, 480–484. Jacobson, N. S., & Truax, P. (1991). Clinical significance: A statistical approach to defining meaningful change in psychotherapy research. Journal of Consulting and Clinical Psychology, 59, 12–19. Johnson, L. N., & Miller, R. B. (2014).Applied statistical analysis and interpretation. In R. B. Miller & L. N. Johnson (Eds.), Advanced methods in family therapy research: A focus on validity and change (pp. 315–328). New York: Routledge. Kim, L. M., Johnson, J. L., & Ripley, J. (2011). A ‘perfect’ storm: Perfectionism, forgiveness, and marital satisfaction. Individual Differences Research, 9, 199–209. Lambert, M. J., Hansen, N. B., Umpress, V., Lunnen, K., Okiishi, J., & Burlingame, G. M., (1996). Administration and scoring manual for the OQ-45.2 (Outcome Questionnaire). Stevenson, MD: American Professional Credentialing Services LLC. Lebow, J. L., Chambers, A. L., Christensen, A., & Johnson, S. M. (2012). Research on the treatment of couple distress. Journal of Marital and Family Therapy, 38, 145–168. Locke, H. J., & Wallace, K. M. (1959). Short marital-adjustment and prediction tests: Their reliability and validity. Marriage and Family Living, 21, 251–255.
October 2014
JOURNAL OF MARITAL AND FAMILY THERAPY
533
Maassen, G. H. (2000). Principles of defining reliable change indices. Journal of Clinical and Experimental Neuropsychology, 22, 622–632. Markman, H. J., & Rhoades, G. K. (2012). Relationship education research: Current status and future directions. Journal of Marital and Family Therapy, 38, 169–200. McLean, L. M., Walton, T., Rodin, G., Esplen, M., & Jones, J. M. (2013). A couple-based intervention for patients and caregivers facing end-stage cancer: Outcomes of a randomized controlled trial. Psycho-Oncology, 22, 28–38. Mead, D., Thurber, S.L., & Crane, B.E. (2003). Spanish translations of a standard assessment battery for marital distress. American Journal Of Family Therapy, 31, 409–412. doi:10.1080/01926180390224003. Miller, R. B., (2000). [National Study of Midlife Marriages Project] Unpublished raw data. Nelson, J. A., O’Brien, M., Blankson, A., Calkins, S. D., & Keane, S. P. (2009). Family stress and parental responses to children’s negative emotions: Tests of the spillover, crossover, and compensatory hypotheses. Journal of Family Psychology, 23, 671–679. Olson, M. M., & Russell, C. S. (2004). Understanding change in conjoint psychotherapy: Inviting clients to comment upon the validity of standardized change scores. Contemporary Family Therapy, 26, 261–278. Ott, C. H., Sanders, S., & Kelber, S. T. (2007). Grief and personal growth experience of spouses and adult-child caregivers of individuals with Alzheimer’s disease and related dementias. The Gerontologist, 47, 798–809. Sabourin, S., Valois, P., & Lussier, Y. (2005). Development and validation of a brief version of the Dyadic Adjustment Scale with a nonparametric item analysis model. Psychological Assessment, 17(1), 15–27. Schramm, D. G., Marshall, J. P., Harris, V., & Lee, T. R. (2005). After ‘I do’: The newlywed transition. Marriage and Family Review, 38, 45–67. Schweinle, W. E., & Ickes, W. (2007). The role of men’s critical/rejecting overattribution bias, affect, and attentional disengagement in marital aggression. Journal of Social And Clinical Psychology, 26, 173–198. Spanier, G. B. (1976). Measuring dyadic adjustment: New scales for assessing the quality of marriage and similar dyads. Journal of Marriage and the Family, 38, 15–28. Sprenkle, D.H. (2012). Intervention research in couple and family therapy: A methodological and substantive review and an introduction to the special issue. Journal of Marital And Family Therapy, 38, 3–29. doi:10.1111/j. 1752-0606.2011.00271.x. Vandeleur, C.L., Fenton, B.T., Ferrero, F.F., & Preisig, M.M. (2003). Construct validity of the French version of the Dyadic Adjustment Scale. Swiss Journal Of Psychology, 62, 167–175. doi:10.1024//1421-0185.62.3.167. Yorgason, J. B., Almeida, D., Neupert, S. D., Spiro, A., & Hoffman, L. (2006). A dyadic examination of daily health symptoms and emotional well-being in late-life couples. Family Relations: An Interdisciplinary Journal of Applied Family Studies, 55, 613–624.
534
JOURNAL OF MARITAL AND FAMILY THERAPY
October 2014
THE DEVELOPMENT OF A RELIABLE CHANGE INDEX AND CUTOFF FOR THE REVISED DYADIC ADJUSTMENT SCALE Shayne R. Anderson, Rachel B. Tambling, Scott C. Huff, and Joy Heafner University of Connecticut
Lee N. Johnson Brigham Young University
Scott A. Ketring Auburn University
The Revised Dyadic Adjustment Scale (RDAS; Busby, Crane, Larson, & Christensen, 1995) is a measure of couple relationship adjustment that is often used to differentiate between distressed and non-distressed couples. While the measure currently allows for a determination of whether group mean scores change significantly across administrations, it lacks the ability to determine whether an individual’s change in dyadic adjustment is clinically significant. This study addresses this limitation by establishing a cutoff of 47.31 and reliable change index of 11.58 for the RDAS by pooling data across multiple community and clinical samples. An individual whose score on the RDAS moves across the cutoff changes by 12 or more points can be classified as experiencing clinically significant change. The effectiveness of therapy can be evaluated in several ways. Howard, Moras, Brill, Martinovich, and Lutz (1996) identified three different types of outcome research, each of which answers different questions about the effectiveness and efficacy of therapy. First, efficacy research answers the question “In general, does this treatment work under controlled laboratory settings?” This can be contrasted with effectiveness research, which answers the question, “In general, does this treatment work under normal clinical conditions?” While both of these questions are important to the field of family therapy, both deal with change at the group level by examining change in the treatment group’s mean scores across time. Neither answers the question of fundamental importance to clinicians—“Is the treatment I am providing helping this particular couple or family?” This third type of more individualized outcome research has direct application for practitioners. It allows clinicians to monitor treatment as it is provided to clients and to accurately assess the extent to which the treatment is influencing client progress on outcomes of interest. By monitoring the provision of treatment and client progress, clinicians obtain evidence that can be used to demonstrate the effectiveness of their work. To track client progress, clinicians need measures that can assess whether clinically significant change is occurring on the constructs of interest (Johnson & Miller, 2014). Of critical importance to family therapists is the question of whether or not a treatment is effective in improving the quality of couple or family relationships. In an attempt to accurately assess relationship quality, a number of self-report measures of dyadic adjustment and satisfaction have emerged including the Dyadic Adjustment Scale (Spanier, 1976), Locke Wallace Marital Adjustment Test (Locke & Wallace, 1959), and the Couple Satisfaction Index (CSI; Funk & Rogge, 2007). One commonly
Shayne R. Anderson, PhD, Rachel B. Tambling, PhD, Scott C Huff and Joy Heafner, Department of Human Development and Family Studies, University of Connecticut; Lee N. Johnson, PhD, Marriage and Family Therapy Program in the School of Family Life, Brigham Young University; Scott. A Ketring, PhD, Department of Human Development and Family Studies, Auburn University. Address correspondence to Shayne R. Anderson, Department Human Development and Family Studies, Unit 1058, University of Connecticut, Storrs, Connecticut 06269-1058; E-mail: [email protected]
October 2014
JOURNAL OF MARITAL AND FAMILY THERAPY
525
used measure of relationship adjustment, the Revised Dyadic Adjustment Scale (RDAS; Busby, Crane, Larson, & Christensen, 1995), has enjoyed both popularity and empirical support, with translations available in French, Portuguese, and Spanish (Hollist et al., 2012; Mead, Thurber & Crane, 2003; Vandeleur, Fenton, Ferrero & Presig, 2003). The RDAS consists of fourteen items extracted from the Dyadic Adjustment Scale (DAS; Spanier, 1976). One of the strengths of the RDAS is that it is able to assess three aspects of dyadic adjustment (consensus, satisfaction, and cohesion) in a valid and reliable way with only 14 items. While similar length scales such as the CSI-16 measure satisfaction more reliably than the RDAS, they are limited to a focus on satisfaction. Researchers and clinicians wishing to assess couple relationships more broadly turn to the RDAS because it has been used in numerous studies with good reliability and is highly correlated with the DAS. Previous research also indicates that the RDAS is successfully able to distinguish between distressed and non-distressed relationships (Busby et al., 1995). Despite the relative strength of the RDAS in comparison with similar instruments, we do not know whether the RDAS can accurately assess whether an individual’s experience of the relationship has changed in a meaningful way as a result of treatment. The purpose of this study is to provide clinicians and researchers with the information to identify whether clients have made meaningful changes in their dyadic adjustment as measured by the RDAS.
ASSESSING CLINICALLY SIGNIFICANT CHANGE Most research in couple therapy uses statistical significance to evaluate whether an intervention is associated with client improvement. While statistical significance has an important role in research, it cannot assess whether differences that occur as a result of treatment are meaningful. It is well documented that given a large enough sample, any difference, regardless of its real world importance can be statistically significant (Cohen, 1994). To assess whether changes are meaningful, we need to move beyond statistical significance to clinical significance. Despite the practical importance of assessing clinical significance, it has not gained sufficient traction in the field of couple and family therapy. While the clinical significance of couple therapy was discussed in the 1990s (Bray & Jouriles, 1995), a similar focus has not been reflected in recent reviews of couple therapy (Lebow, Chambers, Christensen, & Johnson, 2012) or marital enrichment (Markman & Rhoades, 2012), nor was a measure’s ability to assess clinical significance listed as a criteria in evaluating the methodological strength of research in ten substantive areas of family therapy (Sprenkle, 2012). While there are multiple procedures for determining clinical significance, the most widely used is the Jacobson and Truax (1991) method. This method requires that two conditions be met to identify change as clinically significant. First, an individual’s score from pre- to post-test must pass from the dysfunctional to the functional population, typically across a pre-determined cutoff score. Second, the change must be statistically reliable (i.e., large enough to not be due to chance), as measured by a reliable change index (RCI). There have been many attempts to improve upon the Jacobson and Truax method by using various estimate interval approaches. These approaches result in few differences in classification of clinical significance (Atkins, Bedics, McGlinchey, & Beauchaine, 2005) leading to the conclusion that that the classic approach is preferable (Maassen, 2000). The Jacobson and Truax method requires both a cutoff and a RCI to determine whether clients are making clinically significant progress. The RDAS, which lacks an empirically derived cutoff and RCI, cannot currently be used to determine whether clinically significant change has taken place.
THE DEVELOPMENT OF A CUTOFF SCORE ON THE RDAS Crane, Middleton, and Bean (2000) have recommended a cutoff score of 48 for the RDAS. While the cutoff score Crane and associates developed has been useful in helping clinicians and researchers distinguish between distressed and non-distressed couples, it was derived from a regression equation that converts the cutoff on the well-established DAS to an equivalent score on the RDAS. There are two limitations to this approach. First, Crane and associates used a liberal cutoff of 107 on the original Dyadic Adjustment Scale to discriminate between distress and adjustment in their regression equation. This cutoff is typically used to identify well-adjusted individuals. Cutoffs to identify distressed individuals are substantially lower (100: Spanier, 1976; 92: 95: Sabourin, 526
JOURNAL OF MARITAL AND FAMILY THERAPY
October 2014
Valois, & Lussier, 2005). Second, unless the regression equation is perfectly predictive, measurement error can bias the identified cutoff score. A preferable approach is to identify the means of a healthy and distressed population on the measure in question. Jacobson and Truax (1991) suggest the following equation for determining a cutoff score based on two unequal distributions where s represents the standard deviation, M the mean, and 0 or 1 the community and clinical samples: ð1Þ
c¼
s0 M1 þ s1 M0 s0 þ s1
Such a method of determining the cutoff score allows researchers and clinicians to make a determination about which population a given score falls within. Thus, change scores based on cutoff scores that use this method will account for measurement error and unique characteristics of the measure in question.
THE DEVELOPMENT OF A CRITERION FOR CLINICALLY SIGNIFICANT CHANGE ON THE RDAS The second criterion in the Jacobson and Truax (1991) method of establishing clinically significant change is determining that the change is statistically reliable. Classical test theory assumes that an observed score on a measure like the RDAS is a combination of the true score (i.e., actual dyadic adjustment) and measurement error (i.e., the portion of the score that is due to factors not associated with one’s dyadic adjustment). To increase our confidence that changes in scores on the RDAS across administrations represent real changes in an individual’s actual dyadic adjustment, a reliable change index (RCI) must be established. The RCI represents the level of change necessary to be confident that the difference in scores across administrations is not due to chance or error but rather reflects an actual change. To determine a reliable change index, Jacobson and Truax (1991) have suggested the following equation: ð2Þ
RCI ¼
x2 x1 Sdiff
Where x2x1 represents an individual’s change between administrations of the instrument. Sdiff the standard error (SE) of the difference between the two scores, is defined in the following equations: qffiffiffiffiffiffiffiffiffiffiffiffiffiffi ð3Þ Sdiff ¼ 2ðSE Þ2 ð4Þ
SE ¼ s1
pffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi 1 rxx
Sdiff accounts for the variation in reliability of the measure. It represents the standard deviation of the clinical population at intake (s1) and the test–retest reliability (rxx) of the measure in a non-clinical sample. In order for a change to be deemed statistically reliable at the p = .05 level, it must be approximately twice (1.96) the standard error of the difference between the scores. If the change exceeds this threshold, researchers and clinicians can have increased confidence that a difference in scores between administrations of a measure are due to an actual change in the construct of interest rather than error. The goal of this study is to provide clinicians and researchers with the information they need to use the RDAS to determine whether the changes in dyadic adjustment that their clients or research participants are reporting are reliable and clinically significant. In particular, we will answer two questions: 1. What is the cutoff that best distinguishes a clinical population from a community population on the RDAS? 2. How much does an individual’s score on the RDAS need to change to be statistically reliable (i.e., what is the reliable change index for the RDAS)?
October 2014
JOURNAL OF MARITAL AND FAMILY THERAPY
527
METHOD Sample In the absence of normative data, Jacobson and Truax (1991) recommend pooling data across studies that use the RDAS in clinical and community populations. Each study sample is viewed as one of many possible samples drawn from the same distressed or non-distressed population. Pooling data allows for increased heterogeneity of samples and reduces the potential bias using estimates from any given sample could introduce. The data that are pooled in this study come from three sources: Data from community and clinical administrations of the RDAS published in the literature, additional clinical data from three university-based marriage and family therapy training clinics, and a community sample recruited to establish the test–retest reliability of the RDAS. These pooled samples represent 4967 individuals (60% female) in the community samples and 1228 individuals (48% female) in the clinical samples. Table 1 provides the sources of community and clinical samples used in this study. Published studies that have used the RDAS. To identify studies for pooling, we reviewed all studies in the PsychInfo database that cited the article in which the RDAS was developed (Busby et al., 1995). Studies were included if they met the following criteria: (a) The study was peerreviewed research that used the complete RDAS; (b) the study provided sample size, mean, and standard deviation; (c) the study used the English version of the measure; (d) the data in the study were not reported in another study or dataset included in the review; and (e) participants were receiving couple treatment for relational distress or were a part of a community sample. Of the 158 peer-reviewed citations of the RDAS, twenty-two studies met the inclusion criteria. Six studies
Table 1 Sources of Community and Clinical Data Used in Pooled Estimates Community samples
Clinical samples
Blow et al. (2013) Bradford (2012)
Anderson & Miller (2006) Auburn University. MFT Training Clinic Database Busby, Crane, Christensen & Larsen (1995) DeMaria (2005) Gangamma,Bartle-Haring & Glebova (2012)
Bridgett, Burt, Laake & Oddi (2013) Busby, Crane, Christensen, & Larsen (1995) Christie, Meyerowitz, Stanton, Rowland & Ganz (2013) Cook & Jones (2002) Frye-Cox & Hesse (2013) Futris, Campbell, Nielsen & Burwell (2010) Hawkins, Fawcett, Carroll & Gilliland (2006) Hurley, Field, T. & Bendell-Estoff (2012)
McLean, Walton, Rodin, Esplen, & Jones (2013) Olson & Russell (2004) Schweinle & Ickes (2007) University of Connecticut MFT Training Clinic Database University of Georgia MFT Training Clinic Database
Kim, Johnson & Ripley (2011) Miller (2000) Nelson, O’Brien, Blankson, Calkins & Keane (2009) Ott, Sanders & Kelber (2007) Schramm, Marshall, Harris & Lee (2005) Test-Retest Reliability Sample Yorgason, Almeida, Neupert & Hoffman (2006)
528
JOURNAL OF MARITAL AND FAMILY THERAPY
October 2014
provided only clinical data, 15 provided only community data, and one included both a clinical and community sample. Over half of the existing studies (59%) used in this study collected data from both partners of the couple. To account for this non-independence, either the male or female data from each study was chosen at random to include in the pooling. Data were re-analyzed with the other partner’s data to check for bias. Results indicate that choice of partner did not lead to any meaningful changes in the pooled mean or standard deviation. The community and clinical samples were both nearly equally divided between men and women, although the community samples skewed towards more female (63% on average) and the clinical samples skewed towards male (52% on average). The community samples were older (average mean age was 43) than the clinical samples (average mean age was 34). Both samples represented couples from across the United States and were predominately White. Rates of marriage were higher in the community samples (90%) than the clinical samples (77%), although the community samples’ average length of time together was only 2 years longer. Education levels were comparable between the samples, but the community sample reported higher incomes on average. Online supplemental tables provide demographic information, reliability estimates, means and standard deviations for all samples that were pooled in this study. These supplemental tables are included next to the link to this article on the JMFT website. Additional clinical samples. Participants were selected from the archived data of individuals who received couple therapy at one of three family therapy training clinics, located in the Northeast and the Southeast regions of the United States. Individuals, couples, and families with a wide range of presenting problems were treated in the university training clinics by masters or doctoral students enrolled in programs accredited by the Commission on the Accreditation for Marriage and Family Therapy Education. Faculty members in each program provided clinical supervision. Clients were normally seen on a weekly basis and charged on a sliding fee scale based on family income and the number of people supported by that income. Discounted treatment was offered to university students. Clients who consented to participate in research allowed their responses in the clinical assessment packets to be used for research purposes. Prior to the first session, clients completed an intake assessment packet. In addition to demographic information, the client intake assessment packet included several self-report measures of distress, psychological adjustment, and couple relationship functioning including the RDAS. It was important in this study for the clinical sample to represent the population of interest: couples experiencing dyadic distress who are working on ameliorating that distress. After identifying couples receiving therapy, we reviewed the stated presenting problems and removed from the sample those couples who were clearly seeking premarital therapy or who were separated/divorced and working on co-parenting issues. Due to the significant interdependence of couple data, we chose one member from each dyad. If only one partner had complete data on the RDAS, that partner was chosen. When both members of the dyad had complete data we chose the male partner to obtain a more balanced sample. Participants (n = 97) at the Northeastern clinic had a mean age of 31.31 (SD = 10.27). Approximately half were married (48.5%) with an average length of relationship of 4.95 years (SD = 4.21). There were slightly more females than males (54.6%). The sample was predominantly White (73.2%) and well educated (44.2% bachelor’s degree or more), with an average salary range of $40,000 or higher. The mean RDAS score of this sample was 41.74 (11.57). Participants at the first Southeastern clinic (n = 229) were predominantly female (65.9%) and married (57.3%), with an average relationship length of 5.13 years (SD = 5.74). The sample was generally young (M = 30.92, SD = 7.83), White (85%), well educated (62.1% bachelor’s degree or more), and reported a median income range of $20–29,000. The mean RDAS score of this sample was 41.26 (SD = 10.67). Similar to clients at the first two clinics, clients at the second Southeastern clinic (n = 330) reported a mean age of 31.19 (SD = 8.29), were White (76%) and were well-educated, with half reporting receiving at least a bachelor’s degree. This sample was equally divided between men and women (50.6% female) with an average salary range of $30–39,000. Most were married (82.1%)
October 2014
JOURNAL OF MARITAL AND FAMILY THERAPY
529
with an average length of relationship of 7.03 years (SD = 6.06). The mean RDAS score of this sample was 38.39 (SD = 9.92). Test–retest reliability sample. Only one previous study has examined the test–retest reliability of the RDAS. Hawkins, Fawcett, Carroll, and Gilliland (2006) reported 8-week reliability estimates of 0.78 for men and 0.68 for women. These figures likely underestimate the reliability of the RDAS as they were generated during the second and third trimesters from a sample of parents expecting their first child. It is likely that some change in dyadic adjustment occurred during this critical 8-week period, thus lowering the test–retest reliability estimate. To get a more accurate estimate of the true test–retest reliability for the general population, an additional sample was drawn from a larger study examining the measurement properties of several clinical instruments. Participants for the larger study were recruited through advertisements posted to university bulletin boards, online classifieds sites (i.e., the top ten markets on craigslist.org and nationwide on backpage.com), and university student and faculty daily e-newsletters. Participants who completed the study were also given the option to invite their friends to participate through their social media accounts or via e-mail. Of the 1001 people to view the information page at the beginning of the survey, 365 consented to the project and met the eligibility criteria. Of these, 266 completed the first survey. One hundred ninety participants indicated that they were in a committed relationship and completed the RDAS during their first survey. Participants left an e-mail address at the end of the survey that researchers used to contact participants at two, three, and 4 weeks following their first survey. A unique identifier was used to link the first and second surveys while maintaining anonymity. Of the 190 participants who completed the RDAS during the first survey, 88 completed the second survey and provided the unique identifier that matched their first survey. The test–retest period chosen for this study was 2–4 weeks. Excluding participants who completed the survey outside of this time frame resulted in a final sample of 70 participants. Participants were predominantly female (78.6%) White (62.7%) and young, with 66% reporting an age range of 18–29. Most were not married (37.1%) but in a significant committed relationship with a median range of 3–4 years duration. The sample was well educated with approximately 61% reporting having received at least a bachelor’s degree. The median income range of this sample was $40–49,000. The mean RDAS score of this sample was 51.23 (SD = 7.58) at the first administration and 51.24 (SD = 7.81) at the second administration. Measures Revised Dyadic Adjustment Scale. The Revised Dyadic Adjustment Scale (RDAS, Busby et al., 1995) is a shortened version of the Dyadic Adjustment Scale (Spanier, 1976). The RDAS consists of fourteen items designed to measure adjustment in dyadic relationships on three subscales: Consensus, Satisfaction, and Cohesion. The subscales can be summed to create a total score representative of overall dyadic adjustment with a range of 0–69. The total score is used in this study. The RDAS correlates highly with the original DAS (r = .97; Busby et al., 1995) as well as similar measures of dyadic distress (Crane et al., 2000). Busby et al. (1995) factor analysis of the RDAS, also provided support for the construct validity of the measure. In past research, the RDAS has demonstrated strong reliability, with split-half reliabilities ranging from 0.90 to 0.95 for the total scale (Busby et al., 1995). Internal consistency estimates across studies that have used the RDAS have been consistently lower with a mean Chronbach’s alpha of .85 across the samples that provided reliability estimates in this study (range = .76–.91). Cronbach’s alphas from the three university clinics were .86, .87, and .89 while the alpha of the test–retest sample was .80.
RESULTS The 14–28 day test–retest reliability of the test–retest sample was .82. The pooled means and standard deviations for the clinical and community samples on the RDAS are presented in Table 2. Using this information in Jacobson and Truax’s (1991) formula (Equation 1) resulted in a cutoff score of 47.31. Jacobson and Truax’s (1991) method for determining reliable change was used to determine the amount of change in score from pretest to posttest that would be statistically significant at 530
JOURNAL OF MARITAL AND FAMILY THERAPY
October 2014
the p = .05 level. We used the community sample’s test–retest reliability estimate (.82), as well as the standard deviation of the clinical sample at intake presented in Table 2 as inputs for Equations 3 and 4. This resulted in an Sdiff for our sample of 5.91. Using this value in Equation 2 to solve for the level of change that would yield a significant value (i.e., 1.96) resulted in a Reliable Change Index of 11.58.
DISCUSSION This study establishes a cutoff and reliable change index for the RDAS using pooled estimates of the means and standard deviations across multiple community and clinical samples. An individual whose score on the RDAS moves across the cutoff to a score of 48 and changes by 12 or more points from the first to the most recent administration can be classified as experiencing clinically significant change. Changes of at least 12 points that do not cross the cutoff can be classified as reliable improvement or reliable deterioration based on the direction of change. Both the cutoff and the reliable change index can be helpful to clinicians who are attempting to operate a practice in accordance with results-based accountability standards. Clinicians can use these values to measure the effectiveness of treatment for each particular client couple. In addition, these values can assist researchers in establishing the efficacy of various treatments, and in describing clinical change. While it has been important to establish the values, the results of this research suggest that there is room to improve the precision with which the RDAS measures change in dyadic adjustment across time. Of primary concern is the value of the RCI relative to the range of the measure. The RDAS has a range of 70 points. A reliable change index for this measure represents a change of 16.6% of the measure’s range. In contrast, the 16-item version of the Couple Satisfaction Index (Funk & Rogge, 2007), a similar measure to the RDAS but focused solely on satisfaction, has a range of 82 points and an RCI of 9.4 (or 11.5% of the measure’s range; RD Rogge, JL Funk, S Lee & MC Saavedra, unpublished data). Other scales such as the Outcome Questionnaire-45.2 (OQ-45.2; Lambert et al., 1996), a popular measure for examining client progress in individual therapy, have even more narrow bands of reliable change. The OQ-45.2 has a range of 181 points and an RCI of 14 (or 7.8% of the measure’s range). Proportionately, an individual’s change score on the RDAS would have to be 2.1 times greater than the change score on the OQ-45.2 to register as true change. It is likely that the discrepancy in the RCI of these various measures is, in part, an artifact of the length of the measure. Reliability of a measure generally increases as the number of items increases, with increased reliability translating directly into a lower RCI. It is also possible, however, that the lower RCI of these measures comes from the degree of measurement refinement involved in their creation. For example, the CSI was developed using item-response theory, an alternative to classical test theory that results in more reliable measures. The OQ-45.2 was developed specifically as a measure of change in general psychological distress, leading the developers to focus on sensitivity to change throughout its development (Lambert et al., 1996). The reliable change index results in a ceiling for those that can experience reliable change. For example, individuals whose pretreatment RDAS score is 58 cannot reliably improve. While this may not be an issue for clinically distressed relationships, those wishing to use the RDAS to measure improvement at the individual level with non-distressed populations, such as those that present for premarital treatment, may have to search elsewhere for a measure to capture change in
Table 2 Pooled Estimates of Clinical and Community Samples, Cutoff, and RCI of the RDAS
RDAS
October 2014
Couple Therapy Sample
Community Sample
Mp
SDp
n
Mp
SDp
n
Cutoff
RCI
40.58
9.84
1,228
52.95
8.26
4,967
47.31
11.58
JOURNAL OF MARITAL AND FAMILY THERAPY
531
couples’ adjustment. It is possible that additional measurement refinement can make the RDAS a more useful tool for clinicians and researchers wishing to measure change in dyadic adjustment. Such refinement could include improving or replacing items that are particularly unstable across short time periods in community samples. By increasing the short-term temporal stability of the measure in community samples, the test-retest reliability, and consequently the RCI of the measure could be improved. For example, if the reliability were improved from .82 to .92, the RCI would drop almost four points to 7.7. The cutoff score of 47.31 identified in this study is less than one point lower than the score of 48 suggested by Crane et al. (2000). Despite the critique of the original RDAS cutoff, which converted the cutoff on the Dyadic Adjustment Scale to the RDAS using a regression equation, the cutoff proposed in this manuscript is functionally equivalent to the original. The lowest possible non-distressed score is 48 using either cutoff. While the results of this study only alter the cutoff minimally, by using the pooled means and standard deviations across multiple community and clinical samples, we can have more confidence that the cutoff accurately assesses the point at which an individual’s score on the RDAS passes from a clinical to a community population. Limitations The optimal community sample to use for establishing a cutoff score is a normative sample drawn from a national study. In the absence of a normative sample, we have relied on pooled data from multiple clinical and community samples. The replication of samples that are then pooled can provide a better estimate of the true population mean and standard deviation than any one sample alone. However, if a common selection effect occurred across samples, the resulting pooled mean may exhibit bias. This is of most concern in the clinical samples, the majority of which were drawn from university-based family therapy training clinics. It is possible that couples presenting for therapy at training clinics differ in significant ways from those presenting for treatment in other settings. This limitation should be taken into consideration when generalizing these results to broader clinical settings. While a predominantly university-clinic sample does impact the generalizability of the results to settings outside university clinics, it also makes the results particularly applicable to other university clinics where clinical research on couple interventions is often conducted. The sample used to establish the test–retest reliability estimate and the interval between administrations should also be taken into consideration when using the proposed RCI. The RCI identified in this study relies on a 2–4 week test–retest reliability estimate from a convenience sample. While convenience samples are the norm for establishing the test–retest reliability of an instrument, it is possible that the estimate would differ if a representative sample were used. The 2–4 week interval between administrations was chosen to balance recollection bias and the possibility that true change had occurred. Altering this interval would likely alter the test–retest reliability of the measure with shorter intervals increasing the estimate and longer intervals decreasing the estimate. Any changes to the test–retest reliability estimate would have an effect on the resulting RCI. Despite these limitations and the importance of further refining the RDAS suggested earlier, this study has offered clinicians who desire to measure the effectiveness of their clinical work with each client a way to do so. Using the improved cutoff and RCI will also allow researchers who use the RDAS to more accurately gauge whether the interventions they are testing lead to clinically meaningful changes in dyadic adjustment. It is our hope that clinicians and researchers will use this information to document the effectiveness of their clinical practice to provide consumers of therapy with evidence of the effectiveness of a given treatment. Furthermore, we hope that clinicians will use this information to monitor the progress of couples’ dyadic adjustment in therapy, allowing them to modify treatment plans when their client’s relational adjustment is not responding to treatment. We also hope that researchers will expand measurement work beyond initial validity and reliability testing to provide cutoffs, reliable change indices, and other information that will enhance the field’s ability to use formal assessment in clinical practice and clinical research. As researchers and clinicians examine the clinical significance of the work that they do, we are confident that both clients and the field of MFT will benefit.
532
JOURNAL OF MARITAL AND FAMILY THERAPY
October 2014
REFERENCES Anderson, S. R., & Miller, R. B. (2006). The effectiveness of therapy with couples reporting a history of childhood sexual abuse: An exploratory study. Contemporary Family Therapy, 28, 353–366. Atkins, D. C., Bedics, J. D., McGlinchey, J. B., & Beauchaine, T. P. (2005). Assessing clinical significance: Does it matter which method we use? Journal of Consulting and Clinical Psychology, 73, 982–989. Blow, A. J., Gorman, L., Ganoczy, D., Kees, M., Kashy, D. A., Valenstein, M., et al. (2013). Hazardous drinking and family functioning in National Guard veterans and spouses postdeployment. Journal of Family Psychology, 27, 303–313. Bradford, K. (2012). Assessing readiness for couple therapy: The Stages of Relationship Change Questionnaire. Journal of Marital and Family Therapy, 38, 486–501. Bray, J. H., & Jouriles, E. N. (1995). Treatment of marital conflict and prevention of divorce. Journal of Marital and Family Therapy, 21, 461–473. Bridgett, D. J., Burt, N. M., Laake, L. M., & Oddi, K. B. (2013). Maternal self-regulation, relationship adjustment, and home chaos: Contributions to infant negative emotionality. Infant Behavior and Development, 36, 534–547. Busby, D. M., Crane, D. R., Larson, J. H., & Christensen, C. (1995). A revision of the Dyadic Adjustment Scale for use with distressed and nondistressed couples: Construct hierarchy and multidimensional scales. Journal of Marital and Family Therapy, 21, 289–308. Christie, K. M., Meyerowitz, B. E., Stanton, A. L., Rowland, J. H., & Ganz, P. A. (2013). Characteristics of breast cancer survivors that predict partners’ participation in research. Annals of Behavioral Medicine, 46, 107–113. Cohen, J. (1994). The earth is round (p < .05). American Psychologist, 49, 997–1003. Cook, J. L., & Jones, R. M. (2002). Congruency of identity style in married couples. Journal of Family Issues, 23, 912–926. Crane, D. R., Middleton, K. C., & Bean, R. A. (2000). Establishing criterion scores for the Kansas Marital Satisfaction Scale and the Revised Dyadic Adjustment Scale. American Journal of Family Therapy, 28, 53–60. DeMaria, R. M. (2005). Distressed Couples and Marriage Education. Family Relations: An Interdisciplinary Journal of Applied Family Studies, 54, 242–253. Frye-Cox, N. E., & Hesse, C. R. (2013). Alexithymia and marital quality: The mediating roles of loneliness and intimate communication. Journal of Family Psychology, 27, 203–211. Funk, J. L., & Rogge, R. D. (2007). Testing the ruler with item response theory: Increasing precision of measurement for relationship satisfaction with the Couples Satisfaction Index. Journal of Family Psychology, 21, 472–583. Futris, T. G., Campbell, K., Nielsen, R. B., & Burwell, S. R. (2010). The Communication Patterns Questionnaire Short Form: A review and assessment. The Family Journal, 18, 275–287. Gangamma, R., Bartle-Haring, S., & Glebova, T. (2012). A study of contextual therapy theory’s relational ethics in couples in therapy. Family Relations: An Interdisciplinary Journal of Applied Family Studies, 61, 825–835. Hawkins, A. J., Fawcett, E. B., Carroll, J. S., & Gilliland, T. T. (2006). The marriage moments program for couples transitioning to parenthood: Divergent conclusions from formative and outcome evaluation data. Journal of Family Psychology, 20, 561–570. Hollist, C. S., Falceto, O. G., Ferreira, L. M., Miller, R. B., Springer, P. R., Fernandes, C. C., et al. (2012). Portuguese translation and validation of the revised Dyadic Adjustment Scale. Journal of Marital and Family Therapy, 38, 348–358. Howard, K. I., Moras, K., Brill, P. L., Martinovich, Z., & Lutz, W. (1996). Evaluation of psychotherapy: Efficacy, effectiveness, and patient progress. American Psychologist, 51, 1059–1064. Hurley, E. C., Field, T., & Bendell-Estoff, D. (2012). Rejection sensitivity and marital adjustment among military spouses during deployments. Psychology, 3, 480–484. Jacobson, N. S., & Truax, P. (1991). Clinical significance: A statistical approach to defining meaningful change in psychotherapy research. Journal of Consulting and Clinical Psychology, 59, 12–19. Johnson, L. N., & Miller, R. B. (2014).Applied statistical analysis and interpretation. In R. B. Miller & L. N. Johnson (Eds.), Advanced methods in family therapy research: A focus on validity and change (pp. 315–328). New York: Routledge. Kim, L. M., Johnson, J. L., & Ripley, J. (2011). A ‘perfect’ storm: Perfectionism, forgiveness, and marital satisfaction. Individual Differences Research, 9, 199–209. Lambert, M. J., Hansen, N. B., Umpress, V., Lunnen, K., Okiishi, J., & Burlingame, G. M., (1996). Administration and scoring manual for the OQ-45.2 (Outcome Questionnaire). Stevenson, MD: American Professional Credentialing Services LLC. Lebow, J. L., Chambers, A. L., Christensen, A., & Johnson, S. M. (2012). Research on the treatment of couple distress. Journal of Marital and Family Therapy, 38, 145–168. Locke, H. J., & Wallace, K. M. (1959). Short marital-adjustment and prediction tests: Their reliability and validity. Marriage and Family Living, 21, 251–255.
October 2014
JOURNAL OF MARITAL AND FAMILY THERAPY
533
Maassen, G. H. (2000). Principles of defining reliable change indices. Journal of Clinical and Experimental Neuropsychology, 22, 622–632. Markman, H. J., & Rhoades, G. K. (2012). Relationship education research: Current status and future directions. Journal of Marital and Family Therapy, 38, 169–200. McLean, L. M., Walton, T., Rodin, G., Esplen, M., & Jones, J. M. (2013). A couple-based intervention for patients and caregivers facing end-stage cancer: Outcomes of a randomized controlled trial. Psycho-Oncology, 22, 28–38. Mead, D., Thurber, S.L., & Crane, B.E. (2003). Spanish translations of a standard assessment battery for marital distress. American Journal Of Family Therapy, 31, 409–412. doi:10.1080/01926180390224003. Miller, R. B., (2000). [National Study of Midlife Marriages Project] Unpublished raw data. Nelson, J. A., O’Brien, M., Blankson, A., Calkins, S. D., & Keane, S. P. (2009). Family stress and parental responses to children’s negative emotions: Tests of the spillover, crossover, and compensatory hypotheses. Journal of Family Psychology, 23, 671–679. Olson, M. M., & Russell, C. S. (2004). Understanding change in conjoint psychotherapy: Inviting clients to comment upon the validity of standardized change scores. Contemporary Family Therapy, 26, 261–278. Ott, C. H., Sanders, S., & Kelber, S. T. (2007). Grief and personal growth experience of spouses and adult-child caregivers of individuals with Alzheimer’s disease and related dementias. The Gerontologist, 47, 798–809. Sabourin, S., Valois, P., & Lussier, Y. (2005). Development and validation of a brief version of the Dyadic Adjustment Scale with a nonparametric item analysis model. Psychological Assessment, 17(1), 15–27. Schramm, D. G., Marshall, J. P., Harris, V., & Lee, T. R. (2005). After ‘I do’: The newlywed transition. Marriage and Family Review, 38, 45–67. Schweinle, W. E., & Ickes, W. (2007). The role of men’s critical/rejecting overattribution bias, affect, and attentional disengagement in marital aggression. Journal of Social And Clinical Psychology, 26, 173–198. Spanier, G. B. (1976). Measuring dyadic adjustment: New scales for assessing the quality of marriage and similar dyads. Journal of Marriage and the Family, 38, 15–28. Sprenkle, D.H. (2012). Intervention research in couple and family therapy: A methodological and substantive review and an introduction to the special issue. Journal of Marital And Family Therapy, 38, 3–29. doi:10.1111/j. 1752-0606.2011.00271.x. Vandeleur, C.L., Fenton, B.T., Ferrero, F.F., & Preisig, M.M. (2003). Construct validity of the French version of the Dyadic Adjustment Scale. Swiss Journal Of Psychology, 62, 167–175. doi:10.1024//1421-0185.62.3.167. Yorgason, J. B., Almeida, D., Neupert, S. D., Spiro, A., & Hoffman, L. (2006). A dyadic examination of daily health symptoms and emotional well-being in late-life couples. Family Relations: An Interdisciplinary Journal of Applied Family Studies, 55, 613–624.
534
JOURNAL OF MARITAL AND FAMILY THERAPY
October 2014
