This article was downloaded by: [Florida Atlantic University] On: 11 August 2015, At: 05:20 Publisher: Routledge Informa Ltd Registered in England and Wales Registered Number: 1072954 Registered office: 5 Howick Place, London, SW1P 1WG

Psychotherapy Research Publication details, including instructions for authors and subscription information: http://www.tandfonline.com/loi/tpsr20

Why do psychotherapists participate in psychotherapy research and why not? Results of the Attitudes to Psychotherapy Research Questionnaire with a sample of experienced German psychotherapists a

b

Svenja Taubner , Jennifer Klasen & Thomas Munder

c

a

Department of Psychology, Alpen-Adria-University Klagenfurt, Klagenfurt, Austria

b

Department of Psychology, University Kassel, Kassel, Germany

c

Psychologische Hochschule Berlin, Berlin, Germany Published online: 22 Jul 2014.

Click for updates To cite this article: Svenja Taubner, Jennifer Klasen & Thomas Munder (2014): Why do psychotherapists participate in psychotherapy research and why not? Results of the Attitudes to Psychotherapy Research Questionnaire with a sample of experienced German psychotherapists, Psychotherapy Research, DOI: 10.1080/10503307.2014.938256 To link to this article: http://dx.doi.org/10.1080/10503307.2014.938256

PLEASE SCROLL DOWN FOR ARTICLE Taylor & Francis makes every effort to ensure the accuracy of all the information (the “Content”) contained in the publications on our platform. However, Taylor & Francis, our agents, and our licensors make no representations or warranties whatsoever as to the accuracy, completeness, or suitability for any purpose of the Content. Any opinions and views expressed in this publication are the opinions and views of the authors, and are not the views of or endorsed by Taylor & Francis. The accuracy of the Content should not be relied upon and should be independently verified with primary sources of information. Taylor and Francis shall not be liable for any losses, actions, claims, proceedings, demands, costs, expenses, damages, and other liabilities whatsoever or howsoever caused arising directly or indirectly in connection with, in relation to or arising out of the use of the Content. This article may be used for research, teaching, and private study purposes. Any substantial or systematic reproduction, redistribution, reselling, loan, sub-licensing, systematic supply, or distribution in any form to anyone is expressly forbidden. Terms & Conditions of access and use can be found at http:// www.tandfonline.com/page/terms-and-conditions

Psychotherapy Research, 2014 http://dx.doi.org/10.1080/10503307.2014.938256

EMPIRICAL PAPER

Why do psychotherapists participate in psychotherapy research and why not? Results of the Attitudes to Psychotherapy Research Questionnaire with a sample of experienced German psychotherapists SVENJA TAUBNER1, JENNIFER KLASEN2, & THOMAS MUNDER3 Department of Psychology, Alpen-Adria-University Klagenfurt, Klagenfurt, Austria; 2Department of Psychology, University Kassel, Kassel, Germany & 3Psychologische Hochschule Berlin, Berlin, Germany

Downloaded by [Florida Atlantic University] at 05:20 11 August 2015

1

(Received 9 October 2013; revised 18 June 2014; accepted 19 June 2014)

Abstract Objective: Psychotherapy research needs to convince psychotherapists to contribute their time and effort to participate. The present paper describes the development and first results of the Attitudes to Psychotherapy Research Questionnaire (APRQ). Method: The APRQ and additional qualitative questions about exclusion criteria for participation were filled out by a sample of 365 therapists (psychoanalytic, psychodynamic, and cognitive-behavioral) during an Internet-based research survey. Results: A principle component analysis yielded six factors: Benefits, Damage, Legitimation, Self-doubt, Effectiveness Doubt and External Reasons. Age, external reasons and benefits predicted willingness to participate independent of therapeutic school. Discussion: Results show a changing trend to more positive attitudes to psychotherapy research. To be willing to participate in future research, therapists expected high quality designs, financial compensation, and personal gains. Keywords: attitudes to psychotherapy research; psychotherapists; validation; qualitative research methods; factor analysis

The feasibility of psychotherapy research depends on clinicians’ willingness to participate. Especially observational studies need to convince real-life psychotherapists to contribute their time and effort. Furthermore, clinical trials can be improved by including highly experienced psychotherapists and the participation of experienced psychotherapists in research may help to overcome the gap between research and clinical practice (Talley, Strupp, Beutler, 1994). The relation between clinical practice and research has been compromised by reciprocal criticism and prejudice; for example, clinicians doubt clinical relevance of research, whereas researchers question the scientificalness of therapeutic approaches used by clinicians (Elliott & Morrow-Bradley, 1994). The difficult relation between clinical practice and research is also documented by studies showing that clinicians do not read research results (Cohen, 1979; Schachter &

Luborsky, 1998), do not make use of research results (Morrow-Bradley & Elliott, 1986), and rarely initiate research (Prochaska & Norcross, 1983). The question why psychotherapists agree or disagree to participate in psychotherapy research has hardly been addressed in empirical studies. The few existing studies agree that the motivation to participate actively in psychotherapy research is limited. Ward and Richards (1968) report that only 18 from 100 clinicians agreed to participate in their research project. Bednar and Shapiro (1970) asked 16,100 psychotherapists to participate by audiotaping two psychotherapy cases and only 85 psychotherapists (less than 1%) responded. The authors conclude: There is […] increasing reason to believe that the attitude toward psychotherapy research held by the majority of practicing clinicians is not only indifferent as many have long feared, but uncooperative; and by inference, antagonistic (Bednar & Shapiro,

Correspondence concerning this article should be addressed to Svenja Taubner, Department of Psychology, Alpen-Adria-University Klagenfurt, Klagenfurt, Austria. Email: [email protected] © 2014 Society for Psychotherapy Research

Downloaded by [Florida Atlantic University] at 05:20 11 August 2015

2

S. Taubner et al.

1970, p. 323). Wynne, Susman, Ries, Birringer, and Katz (1994) contacted 845 psychotherapists to participate in a process study that included audiotaping of one session and reported that 98% of the clinicians declined participation. Vaughan et al. (2000) reported to have canceled their clinical trial because they were not able to recruit enough patients. The majority of patients rejected study participation after being informed by their therapist. The authors interpreted this effect as a result of the therapists’ dismissive attitude toward psychotherapy research. Two studies investigated therapists’ motives for non-participation by collecting questionnaires or telephone interviews (Bednar & Shapiro, 1970; Vachon et al., 1995). Reasons for non-responding were mainly lack of time and criticism concerning the particular study design or implied methods (e.g., audiotaping; c.f., Table I for details). External reasons for non-participation were also confirmed by Ward and Richards’ (1968) report of a smaller sample of 43 psychotherapists in which 75% refused study participation because of lack of time. Vachon et al. (1995) also asked for aspects that would encourage therapists to participate in research. Again mainly external factors were given, for example, more time, monetary compensation but also research designs without audiotaping. However, 20% of the sample reported a general objection to psychotherapy research. A more recent study may give evidence of a changing trend in the attitudes toward psychotherapy research: Felber and Margreiter (2007) assessed attitudes to psychotherapy research using semi-structured interviews. Results indicated that the 50 participating Austrian psychotherapists were sympathetic toward psychotherapy research and pointed out reasons for participation mainly in the domains of research on effectiveness and quality assurance. Furthermore, about half of the sample was able to name recent studies or research projects. These positive trends are also confirmed by a study from Thurin, Thurin, and Midgley (2012) who investigated 36 mainly

psychoanalytic psychotherapists in France about their motives, conditions, and experiences for participation in a network study using an online Q-sort method. Results underline that psychotherapists perceived positive effects on their own clinical practice and reported no negative effects on their patients by research participation. The authors underline the importance of setting up a peer group as a structural support system for participating psychotherapists which also has been reported by Taubner, Buchheim, Rudyk, Kächele, and Bruns (2012). In sum, knowledge about current attitudes toward psychotherapy research is limited due to outdated research studies or more recent but non-representative and mainly qualitative data. Furthermore, deeper motive structures have not been explored in past research efforts and therefore lack of time is the most prominent motive for non-participation. To design future psychotherapy research that stronger acknowledges psychotherapists’ needs, it seems urgently necessary to know more about the reasons for a possibly dismissive attitude toward research. The aim of the presented study was to fill this gap by deepening knowledge about current motives and attitudes of therapists toward psychotherapy research on a more representative sample of experienced clinicians. Our special interest was to explore attitudes aside from reasons in the external domain by developing the Attitudes to Psychotherapy Research Questionnaire (APRQ), first questionnaire on attitudes toward psychotherapy research. The APRQ may help (i) to explore motive structures that determine participation in research, (ii) to differentiate between different therapeutic schools, (iii) conduct feasibility studies, and (iv) design study designs more appealing to practitioners. The present paper describes the development of the APRQ, first psychometric properties and factor structure. Furthermore, we will report data from the APRQ with a sample of highly experienced German psychotherapists of three different therapeutic schools (psychoanalytic, psychodynamic, and

Table I. Motives for non-responding to psychotherapy research. Bednar and Shapiro (1970) Questionnaire Survey (N = 224) No time (29%) No longer therapist (24.5%) No explicit reason (13.3%) Unsuitable clients, working conditions (8%) Involved in another research (2.7%) Refuse to record therapy sessions (2.7%) Ethical reasons (1.8%) Disagree with project (1.3%)

Vachon et al. (1995) Telephone interview (N = 156) Taping concerns (33%) Time concerns (28%) Clients not appropriate for research (16%) External factors (10%) Research concerns (9%) Impact on therapeutic relationship (5%)

Psychotherapy Research cognitive-behavioral therapists) including qualita‐ tive data on inclusion and exclusion criteria for participation. Method

Downloaded by [Florida Atlantic University] at 05:20 11 August 2015

Construction of the APRQ Following the suggestions from Bühner (2006), we used the data from a qualitative study to construct items for the APRQ inductively. The qualitative study on six psychoanalysts was part of a substudy within a neurobiological research project that focused on the impact of this kind of research on psychoanalysts and psychoanalytic psychotherapies (Taubner et al., 2012). The qualitative study approached the question under which circumstances psychoanalysts would participate in this kind of research and psychotherapy research in general to learn about their attitudes to decide against participation (Protz, Kächele, & Taubner, 2012). Aiming at a broad spectrum of different attitudes, interview partners were selected from a pool of psychoanalysts who had participated in the neurobiological psychotherapy study or who had declined study participation in this particular study, respectively. In total, eight psychoanalysts were asked to participate in the interview, two declined because of confidentiality reasons. In the remaining sample of six licensed psychoanalysts (two men and four women), mean age was 55 years (SD = 3.55) with a mean of 16 years of work experience (SD = 6.01); all psychoanalysts were working full time in private practices. Interviews were semi-structured using a standardized interview protocol, audiotaped, transcribed verbatim, and analyzed by using content analysis (Mayring, 2008) and comparative casuistic (Jüttemann, 2009). The goal of the qualitative analysis was to reduce the material into single condensed statements close to the original wording. In the next step, following the suggestions by comparative casuistic (Jüttemann, 2009), single statements were grouped into superior categories. Results revealed complex and ambivalent attitudes toward psychotherapy

3

research that were connected to needs not only for safety and protection for patients but also for the psychoanalysts themselves (cp., Protz et al., 2012). Selected attitudes are listed in Table II to illustrate unresolved ambivalent attitudes. Four factors as main themes were taken from the qualitative analysis that was generalizable to any psychotherapeutic school: (i) general attitudes toward psychotherapy research (12 items), (ii) perceived therapeutic self-efficacy (11 items), (iii) perceived efficacy of the therapeutic school (eight items), and (iv) protective needs (nine items). In total, 40 items were constructed using the subordinate categories from the comparative casuistic to be tested in the study. Although the interview material was related to psychoanalytic psychotherapies only, items were constructed addressing psychotherapeutic schools in general. The first version of the APRQ was sent to four experts in the field of psychotherapy research to give feedback about the clarity of item phrasing. Based on this feedback, items were revised. Items were scored using a 5-point Likert-scale from 0 (“not at all”) to 4 (“very much”) (compare Appendix 1 for the revised version of the APRQ).

Instruments Additional information on therapists was collected: sex, age, academic degree prior to psychotherapy training, work experience, and therapeutic school. We also asked participants about prior research participation as well as willingness to implement questionnaires and audiotaping therapy sessions. These questions were constructed with either yes or no icons and dummy-coded (yes = 1, no = 0). Willingness to participate in future psychotherapy research was coded using a 3-point Likert-scale with 3 (yes), 2 (maybe), and 1 (no). Two open questions were asked to the participants: (i) “What kind of requirements are essential for you to participate in psychotherapy research?” and (ii) “What would be an exclusion criterion for you to participate in psychotherapy research?”

Table II. Examples of attitudes toward psychotherapy research from qualitative analysis of interviews with six psychoanalysts. Pro-psychotherapy research I hope that psychoanalysis will improve its image and stay within the healthcare system. Objective research bears new possibilities and bears a liberating component. I am interested and curious to learn something new that enriches my therapeutic work. I want share information and make arrangements with the research team.

Contra-psychotherapy research I fear that psychoanalysis could be damaged, if research demonstrates its ineffectiveness. I fear that objective research will not grasp the unique psychoanalytic process and may lead to a devaluation of my work. I do not see any value for the development of psychoanalytic theories and practice. Research and clinical practice should be kept strictly apart from each other.

4

S. Taubner et al.

Procedure

Downloaded by [Florida Atlantic University] at 05:20 11 August 2015

All items were implemented in an Internet-based research survey (“www.psychotherapie-befragung. de”). The survey was conducted online from May 2011 until May 2012. The study was announced with the help of the central and four federal chambers (Hessen, Berlin, Bremen, and Bayern) for licensed psychotherapists (“Psychotherapeutenkammer”) in psychoanalytic psychotherapy (PA), psychodynamic psychotherapy (PD), and cognitivebehavioral therapy (CBT). Participants were offered three gift coupons within a lottery after study completion if they left their email addresses. The online survey was introduced by the following paragraph: In recent years, a growing demand for effectiveness and efficacy studies has been proposed in the field of psychotherapy research. The following questions are designed to highlight the question what kind of attitudes lead to participation in psychotherapy research and what kind of motives may refrain psychotherapists from participation. Please tick what is mostly applicable for you.

Sample The sample consisted of 365 therapists who filled out the online survey, 237 of these were female (66.6%). Age ranged from 28 to 80 years with a mean of M = 52.3 (SD = 10.4) and a Median of 53 years, work experience ranged from 1 to 45 years with a mean of M = 18.5 (SD = 10.5) and a Median of 18 years. Prior to psychotherapy training, 64 participants (18%) had a medical degree, 253 participants (71.1%) had a psychology degree, and 12 (3.4%) were social workers. Twenty-seven participants (7.6%) did not give information on prior degrees. In Germany in 2012, 27,715 licensed psychotherapists were registered of which 22,165 were mainly psychologists and 5550 had a medical degree (20%). The mean age of licensed psychotherapists in 2012 was M = 52.4 years, 69% of licensed psychotherapists in Germany are females (cp., Statistics from the national chambers for psychological and medical psychotherapists in Germany, www.gbe-bund.de, www.kbv.de). Therefore, the present sample resembles the current demographic characteristics of psychotherapists in Germany. Therapeutic schools were represented by 145 PA (40.7%), 126 PD (35.4%), and 85 CBT (23.9%) therapists. Post hoc tests after analysis of variance showed that subsamples of therapeutic schools were significantly different in terms of age (PA > TP > CBT) and work experience (PA > PD, PA > CBT).

Because age and work experience were confounded (r = .83, p = .000), only age was controlled for in analyses including therapeutic school. The proportion of men and women varied significantly across therapeutic schools (χ2 = 6.216, df = 2, p = .045), with a higher percentage of men in the PA subsample (40.7% males) compared to the PD (30.2% men) and CBT (25.9% men) subsamples. Therefore, sex was also controlled for in subsequent analyses with therapeutic school as variable. There were no differences in prior academic degrees between the therapeutic schools. Statistics Firstly, three exploratory principle component analyses (PCA) with different (varimax and promax) or no rotation were conducted to analyze the factor structure of the data-set. Sample adequacy and factorability of the data-set was tested using Kaiser– Meyer–Olkin (KMO) measure and Bartlett Test for Sphericity. Varimax rotation was then chosen because it yielded the lowest number of items with double or no factor loadings. To enhance homogeneity and reliability of the scales, items with sole or double loadings (factor loading .50 and d < .80 (medium effect), and d > .80 (large effect) (Cohen, 1988). Additionally, correlation values were interpreted following

Psychotherapy Research Cohen (1988) with r ≥ .10 (small effect size), r ≥ .30 (medium effect size), and r ≥ .50 (large effect size). Statistical analysis was performed using the SPSS 19.0 (SPSS Inc., Chicago, IL, USA) and STATA 11.2 (StataCorp, College Station, TX, USA). As no missing values occurred, no missing data procedures were used.

Results

Downloaded by [Florida Atlantic University] at 05:20 11 August 2015

Principle Component Analysis Prior to analysis, we tested sample adequacy and factorability of the whole item set using the KMO measure and the Bartlett Test for Sphericity. The KMO measure showed a good sample adequacy with .883 and factorability was given with a high significance of p = .000 in the Bartlett Test for Sphericity. The PCA with varimax rotation revealed two items with double loadings and four items with no factor loadings, a PCA with promax rotation yielded three double and five items with no loadings and finally a PCA with no rotation lead to four items with double and four items with no loadings. Thus, the PCA with varimax rotation was chosen to fit the data-set best. The six items with either double or no loadings were excluded to enhance the homogeneity and reliability of the scales. The number of factors was derived empirically using parallel analysis (Horn, 1965) which compares the number of components that account for more variance than the components derived from random data by extracting eigenvalues from random datasets that parallel the actual data-set with regard to the number of cases and variables (O’Connor, 2000). The parallel analysis leads to a six-factor solution. The Eigenvalue of the sixth factor with 1.513 in the real data-set was higher than the randomly expected value of 1.451, whereas the Eigenvalue of a seventh factor of 1.271 was lower than the randomly expected Eigenvalue of 1.406. A repeated PCA with varimax rotation of the remaining 34 items lead to a six-factor solution (eigenvalues: 8.2, 3.3, 2.3, 1.8, 1.7, and 1.4) accounting for 54.9% of the variance (c.f., Table III). We only report factor loadings higher than .4. Factor 1 encompassed 11 items capturing global positive attitudes to psychotherapy research. Four items described personal benefits in terms of affiliation to researchers and general enjoyment by participation, whereas seven items captured benefits for psychotherapy itself (e.g., psychotherapy research enriches the therapeutic work, initiates development of psychotherapeutic methods, enables new perspectives on therapists’ work, and also improves the

5

quality of psychotherapeutic work). The first factor was labeled Benefits. In contrast, factor 2 captured seven items with global negative attitudes to psychotherapy research. Negative attitudes were described by items that express a strong concern that results from psychotherapy research could be used to harm a certain therapeutic school or harm the individual therapeutic process and patient, respectively. Thus, the second factor was labeled Damage. The third factor comprises five items that mainly describe attitudes in terms of needing psychotherapy research to ensure a scientific basis of a certain therapeutic school and to ensure remaining within the national health system (four items) but also to include clinical perspectives in psychotherapy research (one item). Therefore, in contrast to the Benefits factor, psychotherapy research is valued stronger for its instrumental usefulness than for its contribution to develop psychotherapies. Thus, factor three was labeled Legitimation. Factor four captures four items, expressing the psychotherapist’s fears of being judged, devaluated, found ineffective, and feeling insecure with their therapeutic abilities. This factor was therefore labeled self-doubt. Whereas factor four assembles concerns about individual failures, factor five comprises four items that refer to the therapists’ general doubt in relation to the effectiveness of their specific therapeutic school in comparison to other therapeutic schools. Factor five was thus labeled with the term Effectiveness doubt. Last, a sixth factor concentrates three items that refer to circumstances like time, space, and money that may underlie the decision to participate in psychotherapy research. This factor was labeled External reasons. Internal consistencies of the six factors were mainly good (Cronbach’s alpha ≥ .7), one was excellent α = .91 for Benefits and one only acceptable with α = .59 for Effectiveness doubt. To create a more economic instrument, we suggest using four items in each scale with the highest Cronbach’s alpha and three items for the factor External reasons, respectively (see Appendix 1). Differences in Attitudes toward Psychotherapy Research in Different Therapeutic Schools Half of the sample (50.3%) answered the question if they would participate in psychotherapy research with yes, 37.9% answered maybe, and only 11.8% denied willingness for participation. There were no differences between therapeutic schools in willingness to participate (χ2 = 2.766, df = 4, p = .598). The majority of participants would be willing to give questionnaires to their patients (87.4%) and even almost two-thirds would agree to audiotape sessions (63.2%). Willingness to employ questionnaires to

6

Factor

Downloaded by [Florida Atlantic University] at 05:20 11 August 2015

Item Participation in psychotherapy research opens new perspectives on my clinical work. The results of outcome studies can help to correct my own practice Psychotherapy research gives important impulses for the development of clinical theories. Psychotherapy research improves the quality of psychotherapy. I think that the communication with researchers is stimulating. I think that psychotherapy research is exiting. By participating in psychotherapy research, I can learn something new. Participation in psychotherapy research enables a positive communication among colleagues. Patients benefit from participating in psychotherapy research. The results of psychotherapy research enrich my clinical work. I think that it is important to further develop psychotherapeutic approaches based on scientific evidence.

The intimacy of therapeutic relationships is strongly impaired by research. Accompanying psychotherapy research takes away the patients’ chance to find a place that only deals with him/her. Psychotherapy research intervenes in a disruptive way with clinical treatments. I fear that psychotherapy becomes a more and more standardized method. Psychotherapy research could cause damage to my psychotherapeutic approach. I fear that research results could be used against psychotherapy for political reasons. I worry that psychotherapy research proves psychotherapy to be ineffective.

Psychotherapy research is important for keeping psychotherapy as a part of the healthcare system. Psychotherapy research is important to gain a scientific status for psychotherapy. I see a major need for controlled clinical trials beyond single case reports. Psychotherapy research is important to demystify psychotherapy. I think it is important to strengthen the practitioner’s perspective in psychotherapy research by practitioners participating in research.

1 Benefits (α = .91)

2

3

4

5

6

Communalities

.82 .81 .73

.707 .665 .675

.73 .69 .68 .67 .66

.621 .586 .616 .517 .509

.62 .56 .54

.452 .407 .596 Damage (α = .78) .70 .69

.514 .558

.67 .59 .59 .55 .46

.696 .583 .543 .514 .521 Legitimation (α = .71) 77

.624

.60 .55 .54 .54

.552 .539 .534 .467

S. Taubner et al.

Table III. Item communalities and factor loadings.

Factor

Item

I am afraid to embarrass myself by participating in psychotherapy research. I am afraid of negative evaluation of my work by participating in psychotherapy research. I feel uncertain about how effectiveness of my clinical work is. I sometimes doubt whether I perform my therapeutic approach correctly

My therapeutic approach is outdated in comparison to other schools. I doubt that my therapeutic approach is more effective than other approaches. I am convinced that my therapeutic approach is very helpful for many patients. I am convinced of my therapeutic school’s effectiveness without scientific proof.

Participation in psychotherapy research takes away spare time. Psychotherapy research means unpaid work for me. Participation in psychotherapy research is not possible for me due to lack of time.

1 Benefits (α = .91)

2

3

4

5

6

Self-doubt (α = .71) .78 .76

Communalities

.698 .712

.61 .57

.549 .486 Effectiveness doubt (α = .59) −.69 −.64 .63 .57

.539 .547 .495 .591 External reasons (α = .75) .77 .75 .73

.652 .607 .613

Psychotherapy Research

Downloaded by [Florida Atlantic University] at 05:20 11 August 2015

Table III. (Continued)

7

Note. N = 356. Sex was dummy-coded with 0 = females and 1 = males. Willingness to participate in psychotherapy research (P) was coded yes = 3, eventually = 2, no = 1, willingness to use standardized questionnaires (Q) and to audiotape therapy sessions (A) was dummy-coded with 1 = yes, 0 = no, partial correlations with age and sex as covariates are presented blow the diagonal. *p < .05; **p < .01; ***p < .001.

.10

−.23*** .16** .08 .34***

.37*** −.04 −.03

.24*** .11*

.00 .08 −.05 .24*** .14** .16** .35*** .53*** −.24*** −.48***

−.47*** .53*** .14** .004 −.23*** .21*** −.21*** .06 .02 −.09 −.09 .21*** −.22*** .10* .06 −.02 −.11* −.08 .00 .02 −.23*** −.07 .05 (1) (2) (3) (4) (5) (6)

Benefits Damage Legitimation Self-doubt Effectiveness doubt External reasons

0.00–4.00 0.00–3.43 0.60–4.00 0.00–3.25 0.00–2.75 0.00–4.00

2.32 1.17 3.01 1.47 0.57 2.07

0.72 0.64 0.67 0.52 0.52 0.97

−.13* .05 −.16** .03 −.01 −.01

.32*** −.29*** .16** .02 −.03 −.40***

5 4 3 2 1 Age Range

M

SD

Sex

P

Q

A

Correlations Descriptives

Table IV. Descriptive statistics and raw correlations of key variables.

Downloaded by [Florida Atlantic University] at 05:20 11 August 2015

−.23*** .33*** −.03 .09 .10

S. Taubner et al. 6

8

patients differed significantly between therapeutic schools (χ2 = 10.466, df = 2, p = .005), starting with an agreement of 80.7% of the PA therapists, followed by 90.5% of the PD therapists and 94.1% of the CBT therapists. Agreement for audiotaping sessions also differed significantly between therapeutic schools (χ2 = 18.595, df = 2, p = .000); PA therapists showed the lowest agreement rate to audiotaping (54.5%), 60.3% of the PD therapists agreed, and the highest agreement was again among the CBT therapists with 82.4%. The factor Benefits had the full possible range from 0 to 4 with a mean of M = 2.32 (SD = 0.72) and was associated with sex (females had slightly higher values, t(354) = −2.56, p = .011) and all other scales except Effectiveness Doubt (compare Table IV). Benefits were positively related to Legitimation with large effect size (r = .53, p < .001), willingness to participate (r = .32, p < .001), to use questionnaires (r = .21, p < .001) and to employ audiotape-recordings (r = .21, p > .001) with small to medium effect sizes and Self-doubt with small effect size (r = .14, p < .01). Benefits were negatively associated with Damage (r = −.48, p < .001) and External Reasons (r = −.23, p < .001). Damage ranged from 0.00 to 3.43 with a mean of M = 1.17 (SD = 0.64), was unrelated to sex and age but correlated negatively with willingness to participate (r = −.29, p < .001), to use questionnaires (r = −.22, p < .001) and to use audiotape-recordings (r = −.21, p > .001) with small to medium effect sizes. Higher scores in the factor Damage were also positively related to higher scores in Self-doubt (r = .16, p < .01) with small and External Reasons (r = .33, p < .001) with medium effect size. Female psychotherapists had higher scores for Legitimation (t(354) = −2.99, p = .003), scores of Legitimation for the whole sample ranged from 0.6 to 4.00 with a mean of M = .3.01 (SD = 0.67) and the factor was positively associated with Self-doubt (r = .37, p < .001) with medium effect size. Older psychotherapists had lower scores in Self-doubt (r = −.23, p < .001) which ranged from 0.00 to 3.25 with a mean of M = 1.47 (SD = 0.52). Effectiveness Doubt had the lowest range (0.00–2.75) and lowest mean value of all scales (M = .57, SD = 0.52) and was only associated with Self-doubt (r = .24, p < .001). External reasons had the full range from 0 to 4 with a mean of M = 2.07 (SD = 0.97) and was negatively associated with general willingness to participate in psychotherapy research (r = −.40, p < .001) with medium effect size and the usage of questionnaires (r = −.11, p < .05) with small effect size. An Analysis of covariance (ANCOVA) controlled for sex and age revealed significant differences

9

Psychotherapy Research Table V. Differences in attitudes to psychotherapy research between therapeutic schools. ANCOVAa PA (n = 145) M (SD) 95% CI

PD (n = 126) M (SD) 95% CI

CBT (n = 85) M (SD) 95% CI

Benefit

2.26 (0.68)

2.25 (0.70)

2.52 (0.79)

3.77 (4)

.005

Damage

1.32 (0.63)

1.20 (0.61)

0.89 (0.62)

5.86 (4)

.000

Legitimation

3.21 (0.60)

2.92 (0.64)

2.79 (0.76)

8.41 (4)

.000

Self-doubt

1.69 (0.44)

1.38 (0.53)

1.25 (0.57)

16.93 (4)

.000

Effectiveness doubt

0.55 (0.48)

0.48 (0.52)

0.75 (0.56)

3.89 (4)

.004

External reasons

2.04 (0.96)

2.10 (0.99)

2.08 (0.99)

0.26 (4)

.904

F (df)

p

Significant post hoc tests with effect sizeb CBT > PA (d = .36, p = .041) CBT > PD (d = .37, p = .028) PA > CBT (d = .69, p = .000) PD > CBT (d = .50, p = .002) PA > PD (d = .47, p = .001) PA > CBT (d = .73, p = .000) PA > PD (d = .64, p = .000) PA > CBT (d = .89, p = .000) CBT > PA (d = .39, p = .023) CBT > PD (d = .51, p = .001) –

Downloaded by [Florida Atlantic University] at 05:20 11 August 2015

a Controlled for age and sex; bCohen’s d, Bonferroni adjusted p. PA, Psychoanalytic therapy; PD, psychodynamic therapy; CBT, cognitive-behavioral therapy.

between therapeutic schools for all scales of the APRQ except external reasons (c.f., Table V). Post hoc tests showed that CBT therapists had higher scores in Benefits (small effect sizes) as well as Effectiveness Doubt (small to medium effect sizes), and lower scores in Damage (medium effect sizes) in comparison to the PA and PD therapists. PA therapists had significantly higher scores in Legitimation (small to medium effect sizes) and Selfdoubt (medium to high effect sizes) in comparison to CBT and PD therapists.

(p = .54). As a further confounder, we entered dummy-coded type of treatment, which did not lead to a significant increase of variance explained (ΔR2 = .002, F(4,349) = .171, p = .49) and, thus, was removed from the model. To explore the relation of therapists’ attitudes toward psychotherapy research with their willingness to participate, we entered the APRQ scales into the model. The incremental variance explained was significant (ΔR2 = .217, F(8,345) = 12.323, p < .001). APRQ External Reasons and APRQ Benefit significantly predicted therapists willingness (p < .001 and p = .006, respectively), all other APRQ scales did not predict willingness (ps ≥ .12). The more therapists perceived participation in research as taking away their leisure time, as being unpaid extra work, or as being incompatible with their time schedule, the less willing they were to participate (ß = −.33). The more therapists perceived research participation as something that might improve their therapeutic practice or optimize psychotherapy in general, the more they were willing to participate in research (ß = .17).

Predictors for Participation in Psychotherapy Research In order to predict therapists’ willingness to participate in psychotherapy research, we first entered therapist mean-centered age and sex as potential confounding variables (see Table VI). The model was significant (p = .016) with age as a significant predictor (p = .008). Older therapists were less willing to participate than younger therapist. Therapist sex did not predict willingness to participate

Table VI. Hierarchical regression of predictors of therapists’ willingness to participate in psychotherapy research. Model Step 1: Therapist age and sex Age Sex Step 2: APRQ Benefit Damage Legitimation Self-doubt Effectiveness doubt External reasons

B

SE

T

p

−.008 .023

.003 .039

−2.66 0.61

.008 .54

.165 −.096 .056 −.041 .097 −.230

.059 .061 .062 .074 .064 .036

2.78 −1.57 0.90 −0.56 1.50 −6.46

.006 .12 .37 .58 .13 CBT .26

8.6% [5.7%, 11.5%]

.075

6.9% [4.3%, 9.6%]

.061

4.2% [2.1%, 6.3%] 3.3% [1.5%, 5.2%]

.003 PD > PA > CBT < .001 PA > PD > CBT

Exclusion criterion for research participation Too Much Effort Disturbance of Therapy Process

Researchers’ Background

Insufficient Data Protection

Ethical Concerns Financial Compensation Low Research Quality

Non-experimental Methodology

“too much effort” “more than half an hour per week” “constraints on tailoring therapy to individual clients” “audio- or videotaping of therapy” “no economic and scientific neutrality of researchers” “Researchers without therapeutic experience” “Inadequate anonymization” “my work is evaluated by others that know me” “untreated control group” “high burden for patients” “unpaid extra work” “no financial compensation” “low quality of questionnaires” “design which doesn’t allow valid conclusion” “exclusively quantitative design” “randomized trial”

We did a further analysis to look into potential interactions of APRQ subscales with type of treatment. Re-entering type of treatment and interaction terms with APRQ scales did not lead to a significant increase in variance explained (ΔR2 = .033, F(22,331) = .676, p = .93). Using ordinal logistic regression instead of OLS left the results unchanged.

Qualitative Results Researchers’ answers to the questions “Which conditions must be met for you to participate in psychotherapy research?” and “What would be an

31.8% [26.9%, 36.6]

.82

20.3% [16.1%, 24.4%]

.83

8.0% [5.2%, 10.8%]

.86

7.8% [5.0%, 10.5%]

.26

6.3% [3.7%, 8.8%]

.16

5.0% [2.7%, 7.3%]

.45

4.8% [2.5%, 7.0%]

.27

4.3% [2.2%, 6.3%]

.076

exclusion criterion for participation?” gave further perspectives on their stance toward psychotherapy research. Answers were obtained from 145 psychoanalytic therapists, 125 psychodynamic therapists, and 85 cognitive-behavioral therapists. Overall, 665 answers were coded for question 1 and 400 for question 2. Regarding the first question, several main categories of reasons could be distinguished, most of them differing substantially in the frequency of mentioning (statistically significant differences in percentages are indicated by non-overlapping 95% confidence intervals). For both questions, the same categories were applicable. Due to the different directions of the questions, categories for the first

Downloaded by [Florida Atlantic University] at 05:20 11 August 2015

Psychotherapy Research question were phrased positively, while categories for the second question were phrased negatively. In order to test whether therapists with different therapeutic background used categories differently, we used chi-square tests to test for differences between percentages. For each question, all categories over 5% or more of answers are listed. For all categories mentioned, Table VII lists examples of therapists’ statements, the percentage of statements from categories, and the p value for the test for differences between different therapist groups. The most frequently mentioned condition for participation in research was Manageable Effort, followed by Transparency about Research Operations and Support, Sufficient Data Protection, and Financial Compensation. Further categories were Undisturbed Therapy Process and Relevant Research Question. Psychodynamic therapists also mentioned the condition of High Quality Research. Among psychoanalytic therapists, a further category with more than 5% was Non-experimental Methodology. The most frequently mentioned exclusion criterion for research participation was Too Much Effort. This category was significantly more often used than all other categories. The second most frequently mentioned category was Disturbed Therapy Process, which was mentioned significantly more frequently than Researchers’ Background, Insufficient Data Protection, Ethical Concerns, and No Financial Compensation. Among psychoanalytic therapists, a further category with more than 5% was Low Quality Research. Psychodynamic therapists mentioned an Experimental Design as exclusion criterion. Discussion The aim of the present study was to gain a more representative update on current therapists’ attitudes toward psychotherapy research using quantitative and qualitative measures on a large sample of German licensed psychotherapists. We were interested to understand the structure of attitudes by applying a factor analysis with the APRQ and by analyzing additional qualitative data on motives for and against research participation. Secondly, scales from the APRQ and sociodemographic variables were used to predict future participation. Half of the sample of experienced psychotherapists reported to be generally willing to participate in psychotherapy research, only close to 12% reported not to be willing to join research efforts at all. The remaining 38% were not yet decided and reported that they would participate maybe. Furthermore, the majority of psychotherapists would be willing to employ questionnaires and two-thirds would also

11

allow audiotaping during sessions. In both cases, significant differences existed between therapeutic approaches with CBT therapists being the most and PA therapists being the least agreeing. The number of positive responses was unexpected given the results of prior research and may indicate a change to more positive attitudes toward psychotherapy research and a higher willingness to participate. At the same time, the questions were asked in a hypothetical way without any consequences in terms of a future participation which may have led to a higher agreement than asking for real participation in an upcoming research project. Additionally, a hypothetical study might be imagined as a perfect study, in which participation is more likely than in a realworld study with inevitable shortcomings. Moreover, the sample might be selective in that way, that participating in an online survey attracts psychotherapists with more positive attitudes while psychotherapists with more negative attitudes would not even participate in such a study. However, regarding the fact that half of the sample was either negative or ambivalent to participation in psychotherapy research leads to the assumption that recruitment was successful in combining a sample of psychotherapists with positive and/or negative attitudes. A factor analysis of the APRQ items revealed six different attitudes factors: Benefits, Damage, Selfdoubt, Legitimation, Effectiveness Doubt and External Reasons. Mean values for the whole sample showed highest scores for Legitimation, followed by Benefits and External Reasons. High scores on Legitimation (“pretty much”) may be related to a current debate in Germany about the effectiveness of therapeutic approaches that are accredited in the healthcare system and recent discussion about questioning the effectiveness of these approaches or accrediting other approaches as well. This is supported by the fact that PA therapists have significantly higher scores on Legitimation whose treatment approach was in the center of political discussion regarding the effectiveness of intensive long-term treatments and less Randomized Controlled Trials (RCTs) to show the effectiveness. However, we expect similar findings in other countries with debates on evidence-based treatments and managed care. Psychotherapists partly agreed to see psychotherapy research as beneficial, with CBT-therapists showing higher scores with small effect sizes than the psychodynamic groups. External reasons have been important factors for study participation in past studies on this subject. In the present sample, psychotherapists partly agreed that reasons in the external domain play a role for participation and this

Downloaded by [Florida Atlantic University] at 05:20 11 August 2015

12

S. Taubner et al.

factor was the only one without any school-related differences. Relatively, lower scores for Damages (“little agreement”) in this sample may also contribute to results from newer qualitative studies regarding a positive change in attitudes, in the sense that psychotherapists see higher possible Benefits and less possible Damages when thinking about psychotherapy research. Again the psychodynamic approaches reported significantly higher agreement with possible damages than the CBT group. Although the reluctance of clinicians to participate in therapy research is a problem for research in every therapeutic school (Bednar & Shapiro, 1970; Wynne et al., 1994), psychoanalytic therapists seemed to have the strongest resistances in the past (MorrowBradley & Elliott, 1986). Moreover, Busch et al. (2001) and Taubner et al. (2012) have shown that psychoanalysts are more critical and reluctant regarding therapy research than their own patients. In contrast, the present study supports the assumption of similar motives across different therapeutic approaches and a trend to more positive attitudes to psychotherapy research in comparison to former studies. PA psychotherapists are more critical toward psychotherapy research than CBT psychotherapists and this is also mirrored by PA-specific qualitative results in terms of a strong criticism against experimental research designs. Interestingly, PA therapists rather doubt their individual competence as therapists, whereas CBT therapists have relatively stronger doubts concerning their approach. However, the therapeutic approach did not predict willingness to participate. Instead, age was a significant predictor in the way that older therapists were less willing to join a research project even though age was negatively associated with self-doubt. Using the APRQ scales to predict future participation, only Benefits and External Reasons turned out to be significant. This central result of our study could mean that even though attitudes to psychotherapy research are differentiated in several motives only personal gains and external criteria play a role in deciding for or against participation. This leads to the tentative conclusion that psychotherapists might not respond to rational appeals concerning the legitimation and scienticalness of a therapeutic approach and at the same time that anxieties like damage for patients, self-doubts and effectiveness doubts also do not explain participation. Future research projects should therefore stress the personal and general gains for therapists and their approach and pay most attention to time and money frames to recruit experienced psychotherapists. Additionally, the younger the therapist the more likely is participation which is a challenge for trials aiming to include experienced therapists.

Results from the qualitative analysis can be summarized to grasp two main areas: external conditions of research projects and quality of the research design. Statements concerning external conditions such as Manageable Effort and Financial Compensation validated the results from regression analysis, showing that External Reasons are most prominent factors for participation. The second domain seems equally important and is not captured in the APRQ scales: How therapists determine the quality of research projects. First of all, therapists seem to expect a clear transparency about design and research question including support during the implementation phase. Secondly, ethical concerns play a role regarding data protection and protection of the therapy process. Thirdly, a substantial portion of therapists described to be only willing to spend their time in high quality research that captures their therapeutic work longitudinally with instruments close to the therapeutic process. A further reason not to participate in research concerned the researchers’ background. Therapists would only participate if researchers were perceived as neutral and having clinical experience of their own. Mainly PA therapists stressed the importance of a non-experimental study design to be motivated to participate. In sum, the perfect study in the therapists’ point of view would be led by a clinically experienced researcher with sufficient neutrality toward the outcome of the research (i.e., without strong allegiance to a specific therapeutic school or opinion). A fair amount of time would be dedicated to explain and discuss research aims, design, and implementation between researcher and clinician. Furthermore, the perfect study would include qualitative measures and follow patients longitudinally without disturbing the therapeutic process. Therapists would either be paid for participation or invest only a minimum of time. Major limitations of the present study are related to a participation of only German therapists and only three different therapeutic approaches. Due to the online assessment, we have no information about non-responding and selection bias concerning the sample. Regarding sex, experience, background, and therapeutic approach, the sample can be regarded as representative for Germany. Strengths of the study are the large sample and the mixture of qualitative and quantitative methods which allow insights in motives of therapists and may help to design future research designs which fit the needs of experienced therapists. It seems necessary to include time and money when preparing a new study that aims to integrate clinicians more strongly than in the past (e.g., establishing a research support network (Thurin et al., 2012) or a supervision group (Taubner et al., 2012).

Psychotherapy Research

Downloaded by [Florida Atlantic University] at 05:20 11 August 2015

References Bednar, R. L., & Shapiro, J. G. (1970). Professional research commitment: A symptom or a syndrome. Journal of Consulting and Clinical Psychology, 34, 323–326. doi:10.1037/h0029339 Bühner, M. (2006). Einführung in die Test- und Fragebogenkonstruktion [Introduction to test and questionaire construction] (2., aktualisierte Auflage). München: Pearson Studium. Busch, F. N., Milrod, B. L., Rudden, M., Shapiro, T., Roiphe, J., Singer, M., & Aronson, A. (2001). How treating psychoanalysts respond to psychotherapy research constraints. Journal of the American Psychoanalytic Association, 49, 961–983. doi:10. 1177/00030651010490030601 Cohen, L. H. (1979). The research readership and information source reliance of clinical psychologists. Professional Psychology 10, 780–785. doi:10.1037/0735-7028.10.6.780 Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Lawrence Erlbaum. Cohen, J., Cohen, P., West, S. G., & Aiken, L. S. (2003). Applied multiple regression/correlation analysis for the behavioral sciences (3rd ed.). New York, NY: Routledge. Elliott, R., & Morrow-Bradley, C. (1994). Developing a working marriage between psychotherapists and psychotherapy researchers: Identifying shared purposes. In P. F. Talley, H. H. Strupp, & S. F. Butler, (Eds.), Psychotherapy research and practice. Bridging the gap (pp. 124–142). New York, NY: Basic Books. Felber, M., & Margreiter, U. (2007). Einstellung von Psychotherapeut(inn)en zu Psychotherapieforschung [Attitudes of psychotherapists to psychotherapy research]. Psychotherapie Forum, 15, 192–196. doi:10.1007/s00729-007-0214-1 Horn, J. L. (1965). A rationale and test for the number of factors in factor analysis. Psychometrika, 30, 179–185. doi:10.1007/BF0 2289447 Jüttemann, G. (2009). Komparative Kasuistik. Die psychologische Analyse spezifischer Entwicklungsphänomene [Comparative casuistic. The psychological analysis of specific developmental phenomena]. Lengerich: Pabst. Mayring, P. (2008). Qualitative Inhaltsanalyse. Grundlagen und Techniken [Qualitative content analysis. Basics and techniques]. Weinheim: Beltz. Morrow-Bradley, C., & Elliott, R. (1986). Utilization of psychotherapy research by practicing psychotherapists. American Psychologist, 41, 188–197. doi:10.1037/0003-066X.41.2.188 O’Connor, B. P. (2000). SPSS and SAS programs for determining the number of components using parallel analysis and Velicer’s MAP test. Behavior Research Methods, Instruments, and Computers, 32, 396–402. doi:10.3758/BF03200807

13

Prochaska, J. O., & Norcross, J. C. (1983). Contemporary psychotherapists: A national survey of characteristics, practices, orientations, and attitudes. Psychotherapy: Theory, Research and Practice, 20, 161–173. doi:10.1037/h0088487 Protz, J., Kächele, H., & Taubner, S. (2012). Die Ambivalenz mit der Therapieforschung – Beweggründe und Erfahrungen von Psychoanalytikerinnen und Psychoanalytikern [Ambivalence with psychotherapy research. Attitudes and experiences from psychoanalysts]. Forum der Psychoanalyse, 28, 67–88. doi:10.1007/s00451-011-0081-9 Schachter, J., & Luborsky, L. (1998). Who’s afraid of psychoanalytic research? Analysts attitudes towards reading clinical versus empirical research papers. International Journal of Psychoanalysis, 79, 965–969. Talley, P. F., Strupp, H. H., & Beutler, S. F. (Ed.). (1994). Psychotherapy research and practice. Bridging the gap. New York, NY: Basic Books. Taubner, S., Buchheim, A., Rudyk, R., Kächele, H., & Bruns, G. (2012). How does biological research influence psychoanalytic treatments? Observations from the Hanse-Neuro-Psychoanalysis-Study. American Journal of Psychoanalysis, 72, 269–286. doi:10.1057/ajp.2012.17 Thurin, J.-M., Thurin, M., & Midgley, N. (2012). Does participation in research lead to changes in attitudes among clinicians? Report on a survey of those involved in a French practice research network. Counseling and Psychotherapy Research, 12, 187–193. doi:10.1080/14733145.2012.696122 Vachon, D. O., Susman, M., Wynne, M. E., Birringer, J., O lshefsky, L., & Cox, K. (1995). Reasons therapists give for refusing to participate in psychotherapy process research. Journal of Counseling Psychology, 42, 380–382. Vaughan, S. C., Marshall, R. D., Mackinnon, R. A., Vaughan, R., Mellman, L., & Roose, S. P. (2000). Can we do psychoanalytic outcome research? A feasibility study. International Journal of Psychoanalysis, 81, 513–527. doi:10.1516/0020757001599933 Ward, C. H., & Richards, J. C. (1968). Psychotherapy research: Inertia, recruitment, and national policy. American Journal of Psychiatry, 124, 1712–1714. Wynne, M. E., Susman, M., Ries, S., Birringer, J., & Katz, L. (1994). A method for assessing therapists’ recall of in-session events. Journal of Counseling Psychology, 41(1), 53–57. doi:10. 1037/0022-0167.41.1.53

14

S. Taubner et al. Appendix 1. Attitudes to Psychotherapy Research Questionnaire (APRQ)

Table A1. Attitudes to psychotherapy research (please tick where applicable).

Downloaded by [Florida Atlantic University] at 05:20 11 August 2015

Not at all Participation in psychotherapy research opens new perspectives on my clinical work. The intimacy of therapeutic relationships is strongly impaired by research. Psychotherapy research is important for keeping psychotherapy as a part of the healthcare system. I am afraid to embarrass myself by participating in psychotherapy research. My therapeutic approach is outdated in comparison to other schools. Participation in psychotherapy research takes away spare time. The results of outcome studies can help to correct my own practice. Accompanying psychotherapy research takes away the patients’ chance to find a place that only deals with him/her. Psychotherapy research is important to gain a scientific status for psychotherapy. I am afraid of negative evaluation of my work by participating in psychotherapy research. I doubt that my therapeutic approach is more effective than other approaches. Psychotherapy research means unpaid work for me. Psychotherapy research gives important impulses for the development of clinical theories. Psychotherapy research intervenes in a disruptive way with clinical treatments. I see a major need for controlled clinical trials beyond single case reports. I feel uncertain about how effectiveness of my clinical work is. I am convinced that my therapeutic approach is very helpful for many patients. Participation in psychotherapy research is not possible for me due to lack of time. Psychotherapy research improves the quality of psychotherapy. Psychotherapy research could cause damage to my psychotherapeutic approach. Psychotherapy research is important to demystify psychotherapy. I sometimes doubt if I perform my therapeutic approach correctly. I am convinced of my therapeutic school’s effectiveness without scientific proof.

A little

Partly

Pretty much

Very much