http://informahealthcare.com/bij ISSN: 0269-9052 (print), 1362-301X (electronic) Brain Inj, 2014; 28(12): 1610–1616 ! 2014 Informa UK Ltd. DOI: 10.3109/02699052.2014.934921
ORIGIANL ARTICLE
Efficacy of memory rehabilitation therapy: A meta-analysis of TBI and stroke cognitive rehabilitation literature Madison Elliott & Frederick Parente
Brain Inj Downloaded from informahealthcare.com by Flinders University of South Australia on 12/27/14 For personal use only.
Towson University, Towson, MD, USA
Abstract
Keywords
Objective: To examine the efficacy of cognitive rehabilitation strategies specifically designed to improve memory after traumatic brain injury (TBI) and stroke vs. memory improvement with the passage of time. Design and methods: A meta-analysis was performed on 26 studies of memory retraining and recovery that were published between the years of 1985 and 2013. Effect sizes (ESs) from each study were calculated and converted to Pearson’s r and then analysed to assess the overall effect size and the relationship among the ESs, patient demographics and treatment interventions. Results: Results indicated a significant average ES (r ¼ 0.51) in the treatment intervention conditions, as well as a significant average ES (r ¼ 0.31) in the control conditions, in which participants did not receive any treatment. The largest ESs occurred in studies of stroke patients and studies concerning working memory rehabilitation. Conclusions: Results showed that memory rehabilitation was an effective therapeutic intervention, especially for stroke patients and for working memory as a treatment domain. However, the results also indicated that significant memory improvement occurred spontaneously over time.
Cognitive rehabilitation, memory, meta-analysis, neurorehabilitation, remediation, stroke, TBI, working memory
Introduction to brain injury At least 5.3 million US citizens live with a permanent TBI- or stroke-related disability. Health professionals consider TBI to be one of the most debilitating injuries relative to other neurological disorders [1, 2]. Mar et al. [3] detailed the devastating societal and individual impact of neurological trauma. Many factors contribute to a person’s recovery trajectory and, in the past decade, rehabilitation of cognitive processes has become significant in the treatment course [4, 5]. Although cognitive rehabilitation is increasingly popular, clinicians currently face major difficulties when identifying and treating cognitive impairments caused by brain injury or stroke. This is because brain-injured populations typically are a heterogeneous group with a multitude of deficits [6]. Evidence-based treatment guidelines are only as good as the research they rely on and published rehabilitation studies tend to be both elaborate and ambiguous in nature. Several professional committees, including members of the National Institutes of Health (NIH) and the Brain InjuryInterdisciplinary Special Interest Group (BI-ISIG) of the American Congress of Rehabilitation Medicine, have
Correspondence: Madison Elliott, Psychology Department, Towson University, 8000 York Road, Towson, MD 21252, USA. E-mail: [email protected]
History Received 15 May 2013 Revised 14 March 2014 Accepted 6 June 2014 Published online 21 July 2014
attempted to amalgamate this literature via published rehabilitation practice guidelines for TBI and stroke. These guidelines are based on findings from recently published systematic reviews, which generally examine a wide body of published studies that include diverse research designs [7]. Systematic reviews construct valuable inferences about the efficacy of rehabilitation strategies; but what they achieve in variety, they also lack in statistical rigour. Meta-analyses must exclude studies that do not provide group data for effect size (ES) and, therefore, cannot examine as many publications as a systematic review. Nevertheless, meta-analyses are becoming increasingly popular in review methodology in psychological and brain sciences and are recognized as a statistically reliable manner of testing objective observations across studies [8]. The purpose of the present investigation is to provide a meta-analytic review of cognitive rehabilitation strategies for the domain of memory in individuals with TBI and stroke. The authors’ goal is to contribute to future guidelines, as well as augment existing qualitative review findings from memory rehabilitation literature. Cognitive rehabilitation of memory Cognitive Rehabilitation Therapy (CRT) for brain injury and stroke is diverse, with numerous treatment and theoretical models. Treatments can be process-specific, aimed at improving overall performance of a given activity, skill-based,
Brain Inj Downloaded from informahealthcare.com by Flinders University of South Australia on 12/27/14 For personal use only.
DOI: 10.3109/02699052.2014.934921
motor-based or restricted to a certain area of cognition like attention, language, executive function or memory. Overall goals of treatment and treatment implementation are as varied as the treatments themselves [9]. This rehabilitation is broad because of the unique nature of every brain injury. In the case of memory impairment, long-term, short-term and/or working memory can be negatively affected. Rehabilitation tasks and their outcome measures are often developed to improve or evaluate function within a specific type of memory [10]. Memory impairment is commonly reported after brain injury or stroke and individuals who experience trauma localized in the temporal lobes, hippocampus and amygdala are particularly susceptible to memory impairment [10]. However, the locus of the brain injury is not the sole determinant of memory problems; many factors contribute to each person’s unique outcome post-brain injury [2]. To explain the varying nature of TBI, Kay [4] and McCrea et al. [5] have proposed integrated, multi-factorial neuropsychological models of brain injury outcome, including the determination of factors such as neurological health, psychological health, physiological health, as well as their interactions with individual subjective or objective cognitive functions. Various behavioural treatment strategies for memory rehabilitation have proven to be efficacious, such as the internet-based tasks used in Bergquist et al. [11]. Other forms of computerized training [12], awareness questionnaires [13] and motive or incentive-based tasks [14] have also resulted in effective treatment. However, relatively few articles have been published that have documented successful pharmaceutical treatments for memory enhancement [15]. The typical behavioural rehabilitation efficacy study evaluates memory differences for a control group vs. experimental group, pre- and post-intervention design. Outcome measures vary greatly and include neuropsychological batteries, motor-function tests, brain imaging and various self-report instruments. Observer versions of the self-report measures have been administered to participants’ family members, professional supervisors or close friends to obtain a more diverse understanding of their daily behavioural functioning [2]. Although there are published reports of potentially effective treatments, there is no conclusive evidence of an effective standardized treatment programme for memory dysfunction. Most treatments produce improvement in memory functioning, but it is unclear whether some treatments are more efficacious than others [16]. Furthermore, there is little research to document enduring improvement over time for any treatment modality. There is some evidence, however, for treatment efficacy that exceeds the level of improvement that occurs with the passage of time. In Rohling et al.’s [8] meta-analytic re-examination of Cicerone et al.’s [10, 17] review, the authors found a larger effect size for memory improvement in TBI and stroke patients after clinical rehabilitation than in TBI and stroke patients who received no treatment. Memory rehabilitation efficacy CRT after TBI and stroke has been the topic of a number of major reviews evaluating treatment effectiveness [8, 10,
Efficacy of memory rehabilitation therapy
1611
17–19]. In their extensive review, Cicerone et al. [10, 17] evaluated 118 studies that used CRT, classifying them first in terms of methodological quality and then summarizing consistency of findings. The highest ranked studies were those including randomized control trials (RCTs) (I), followed by studies including non-randomized control prospective or retrospective case-controls (II). The lowest ranked studies were those not including control groups or those that used data from case studies of single subjects (III). The study also categorized the research into various domains: attention, vision, perception, apraxia, language and communication, working memory, executive functioning, problem-solving and awareness and comprehensive holistic cognitive rehabilitation. Some of the cognitive rehabilitation domains, such as attention training, visual-spatial training and languagebased interventions produced larger treatment effects than others [17, 20]. More recently, Rohling et al. [8] performed a quantitative meta-analysis of a select, well-controlled, literature set that derived from Cicerone et al. [10, 17]. Rohling et al. [8] concluded there was ‘modest qualitative support’ for CRT efficacy, as asserted by Cicerone et al. [10, 17] asserted in their original paper. Rohling et al. [8] found sufficient evidence that attention training after TBI, language and visual-spatial training for symptoms of stroke were effective. The authors concluded that, out of five major treatmenteffectiveness findings in Cicerone et al. [10, 17], their metaanalysis results only supported three: attention-training, language-based training and visual-spatial training. Rohling et al. [8] cautioned readers that Cicerone et al. [10, 17] did not adequately estimate re-test effects from studies with uncontrolled designs. One of the two major claims Rohling et al. [8] questioned was the efficacy of memory rehabilitation. Although their data showed a moderate effect size (ES) for memory rehabilitation, it was not significant. This suggested that general neuroplasticity and natural recovery were responsible for much of the documented memory recovery, even in the treatment conditions. Due to the lack of significant ESs, the authors also could not definitively say that the memory rehabilitation had the desired effect. However, very few of the 115 articles evaluated in Rohling et al. [8] involved memory rehabilitation, so it is difficult to derive any conclusive evidence about the overall efficacy of memory therapy. The present meta-analysis reviewed studies published in 1985–2013 about memory rehabilitation in participants who suffered a TBI or stroke, in order to build upon the aforementioned findings by Cicerone et al. [10, 17] and Rohling et al. [8], regarding the domain of memory rehabilitation. The primary goal of the present meta-analysis was to update Rohling et al.’s [8] work by adding new studies. A secondary goal was to investigate new factors that may contribute to the recovery effects. Many neuro-rehabilitation techniques have changed and advanced since the Rohling et al. [8] meta-analysis. However, the Rohling et al. [8] methodology for meta-analytic research in this area is exemplary. Therefore, the methods for the current study were adapted from Rohling et al. [8]. These procedures include adding the most recent publications, using the most rigorous and careful methods of the four major
1612
M. Elliott & F. Parente
reviews listed and replicating the Rohling et al. [8] selection criteria. A re-examination of new memory rehabilitation studies, and those in Rohling et al.’s [8] database, revealed several new variables that were significant factors during a patients’ recovery. The following variables were identified based on their prevalence and statistical availability: average age of the patients (paediatric vs. adult), year of publication, type of brain injury, type of memory targeted by the rehabilitation strategy, type of intervention (computerized vs. not-computerized) and sample size.
Brain Inj Downloaded from informahealthcare.com by Flinders University of South Australia on 12/27/14 For personal use only.
Meta-analysis and effect size measures A meta-analytic review is a manner of statistically aggregating and evaluating previously conducted studies. Generally, the purpose of a meta-analysis is to examine the overall effect of a specific variable in available literature. Meta-analyses use research studies as a unit of measure to evaluate central tendency and variability of the ESs across the sample of chosen studies. Rohling et al. [8] used a widely accepted procedure extracting ESs from published studies; the ESs measure of the amount of variance the effect controls in a given experiment. Their meta-analysis was a quantitative evaluation of the extent to which covariates of interest control significant portions of the ESs in the study sample. According to the standards used by Rohling et al.’s [8], the overall effect of the variable is considered reliable if the average ES is significantly different from zero. The goal of the present study was to follow Rohling et al.’s [8] example and to reduce the relevant statistical information in each published study to standard units of ES and then evaluate whether the ESs co-vary with any of several other variables that describe the conditions of the original experiments.
Method Sample of studies Studies were identified using combinations of the search terms: ‘working memory’, ‘rehabilitation’, ‘remediation’, ‘memory’, ‘training’, ‘brain injury’, ‘TBI’ and ‘stroke’ in the following research databases: Medline, Pub Med, PsycINFO, PsycArticles and Google Scholar. Once pertinent articles were identified through these databases, their reference lists were reviewed to locate other potentially relevant studies for inclusion. Based on Rohling et al.’s [8] criteria, excluded studies (1) did not contain a memory-rehabilitative intervention; (2) merely described treatment approaches or theories; (3) were review articles; (4) presented unspecified or unmeasured interventions; (5) lacked a diagnosis or assessment of TBI or stroke; (6) were case studies of a single participant with no empirical data; (7) were non-peer reviewed articles; (8) included exclusively-pharmacological interventions; and (9) were not written in English. Rohling et al. [8] mention that single papers may include multiple analysable studies, with ‘unique non-overlapping samples of participants’ (p. 22). In total, 26 published memory rehabilitation studies met selection standards and could be used for the analysis. Fourteen of the studies compared a treatment intervention condition to a control group.
Brain Inj, 2014; 28(12): 1610–1616
The other 12 studies contained a treatment intervention, but no control condition. Studies not cited in article, but included in the meta-analysis are listed [30–51]. Procedural analysis Meta-analytic procedures using the MIX 2.0 for Microsoft Excel meta-analysis software package were used to evaluate the ESs computed from the available statistics in each published study. These ESs (calculated as the Pearson’s r) were subsequently analysed to determine whether the average ESs were significantly different from zero. The researchers chose to use r because of several advantages listed in Rosenthal and Dimatteo [21]. These authors stated that converting to r is advantageous because this ES, in its point biserial form, better represents the relationship between levels of the independent variable and dependent variable scores. Additionally, converting a continuous r to dichotomous d results in a loss of statistical information. Researchers may use degrees of freedom contrast techniques to analyse trends across a greater number of groups with r and it requires no computational adjustments when examining different types of t-test samples. There is also a well-accepted public familiarity with r and its practical importance is established in the field of psychology [21]. Magnitude of ES (small, moderate, large) was interpreted according to the guidelines published in Cohen [22]. ESs from control conditions estimated how much memory improvement was not attributable to an experimental intervention, but instead attributable to the passage of time and non-specific neuroplasticity. ESs from intervention conditions were subsequently analysed and used to estimate how much memory improvement could be attributed to a cognitive rehabilitation effort. The final analytic procedure compared the first (control analysis) to the second (intervention analysis) in order to determine whether cognitive rehabilitation interventions yield significantly larger ESs compared to control conditions, where no intervention has taken place. The Q statistic was calculated to evaluate the significance of the overall effect size and to assess the significance of the difference between the treatment ES once the control ES was removed. The Q statistic and forest plots were also calculated initially to assess the homogeneity of the ESs across the various studies. The observed heterogeneity of ESs in both control and intervention group conditions was used to justify the selection and use of a random effects model for the analysis. This model allows for comparisons to be generalized beyond studies in the current sample and studies identical to those in the current sample [21]. A random effects model was considered superior to a fixed effects model for analysing the current set of studies because fixed effect models are used when the sample of studies are considered homogenous and of the same population [23]. Although MIX 2.0 analytic procedures weighted ESs by their sample size, Begg’s Test statistics also were calculated to assess whether or not sample sizes affected the reported ESs. The degree of dissemination bias, i.e. the extent to which the published studies over-estimate the size of the effect, was also examined using a funnel plot. Covariates of the ESs, including the average age of the patients (paediatric vs. adult), ratio
DOI: 10.3109/02699052.2014.934921
of males to females in the sample, year of publication, type of brain injury, type of memory targeted by the rehabilitation strategy, type of intervention (computerized vs. notcomputerized) and sample size were also evaluated to determine whether or not they predicted memory recovery in either group.
Results
Brain Inj Downloaded from informahealthcare.com by Flinders University of South Australia on 12/27/14 For personal use only.
Control conditions analysis The average effect size for this condition of r ¼ 0.31 was significant (Z ¼ 10.00, p50.05), which indicated a moderate and significant improvement in memory, that was not attributable to a rehabilitation intervention. Figure 1 is a forest plot displaying the ESs for all included control studies. The plot displays the ES and 95% confidence interval for each included study. The extent to which the horizontal lines overlap is an indication of the homogeneity of the ESs across studies. This visual representation of the data demonstrates that the studies were generally not homogenous in their reported ESs, with two possible outliers. The outliers may be due to different methodology or sample characteristics, as suggested by Control Group Q statistic and selectivity funnel plot results. A significant Q statistic (73.259, p ¼ 0.00) also indicates that the studies included are heterogeneous. The I2 statistic estimates the percentage of within-study variance among the studies due to non-random factors. The I2 statistic 82.25% also suggests heterogeneity among the ESs. It is important to note that when k520, as in the present analysis, both the Q and I2 tests should be interpreted with caution [24]. Nevertheless, this statistic suggests that it may be worthwhile to explore characteristics of the various experiments that may
Figure 1. Forest plot of ESs in control group studies.
Efficacy of memory rehabilitation therapy
1613
have contributed to the inconsistency of methods used among the control studies. Begg’s Test statistics were computed to determine whether publication bias was present in the sample of studies used. Smaller samples with fewer degrees of freedom often produce inflated ESs and may indicate the existence of unpublished studies with lower ESs. The Begg’s statistics computed on these data indicated the presence of publication bias among the chosen studies. Funnel plots can be used to illustrate dissemination bias. If there is little or no dissemination bias, a funnel plot will show all the dots distributed randomly around the synthesis estimate line and they will also fall between the confidence interval funnel lines on each side of the plot. The funnel plot from the present analysis, as seen in Figure 2, indicates the presence of dissemination bias in control condition studies. This also means that there is a possibility of unpublished studies with lower ESs existing and reinforces the Begg’s test results that published studies contain inflated ESs which are possibly over-estimating the effect of recovery without an intervention condition. Intervention conditions analysis The average effect size for this group of studies was r ¼ 0.51 (Z ¼ 20.62, p50.05), which was significantly different from zero. The confidence interval for the overall ES (0.462–0.558), did not include the average ES for the control studies (r ¼ 0.31). The results, therefore, indicated that, overall, a moderate and significant improvement in memory was apparent in the cognitive measures used in these studies. This change in memory function cannot be attributed solely to the passage of time. Figure 3 is a forest plot for the intervention condition studies. The visual representation of the data here demonstrates intervention studies were somewhat less heterogeneous than the controls in their reported ESs. These outliers may be due to different methodologies or publication bias, as suggested by the Intervention Group Q statistic and selectivity funnel plot results. The significant Q statistic indicated significant heterogeneity of the ESs (Q ¼ 81.50, p50.05). The I2 statistic of
Figure 2. Funnel plot for control group studies.
Brain Inj Downloaded from informahealthcare.com by Flinders University of South Australia on 12/27/14 For personal use only.
1614
M. Elliott & F. Parente
Figure 3. Forest plot of ESs in intervention group studies.
Brain Inj, 2014; 28(12): 1610–1616
group conditions indicated that studies of stroke rehabilitation produced significantly larger ESs compared to mixed brain injury and TBI studies (p50.05). Confidence intervals were also used to assess ES differences between the intervention (experimental group) and the control group in studies that included both. The discrepancy between these confidence intervals indicated a significant difference between the experimental and the control group ESs (p50.05). A third confidence interval analysis indicated that studies of working memory produced significantly higher ESs relative to studies of other memory processes. There was no significant difference in ESs for intervention studies (experimental groups) that contained a control group vs. intervention studies that did not contain a control group, for average age of the patients in control and intervention studies (paediatric vs. adult) or type of intervention (computerized vs. not-computerized). There was no significant correlation between ES and the ratio of males-to-females in the sample or for the year of publication in control and intervention studies. Post-hoc correlation analyses investigating the relationship between dependent variables in the control and intervention conditions and ESs did not reveal any significant relationship (p40.05) between the ESs and the number of dependent variables in either the control or the experimental conditions. Additional post-hoc correlation analyses show that there was no significant correlation (p40.05) between sample sizes and ESs for intervention (experiment) group studies, but that there was a significant correlation (p50.05) between sample sizes and ESs for control group studies. There were no other significant covariate/ES correlations.
Discussion
Figure 4. Funnel plot for intervention studies.
69.31% indicates that there was somewhat less heterogeneity than the control conditions where I2 ¼ 82.25%. These statistics should be considered more reliable for the intervention analysis (k ¼ 26) according to sample size criteria in Higgins et al. [24]. The Begg’s test statistics computed on these data showed that dissemination bias was present in the sample of studies used for the analysis. The intervention analysis funnel plot (Figure 4) is an additional indication of the presence of dissemination bias in intervention studies. This shows that, similar to published control studies, published intervention studies have inflated ESs and may be over-estimating the effect of recovery due to rehabilitation. Two of the study covariates predicted the ESs. The 95% confidence intervals computed on the ESs in the intervention
This meta-analytic review investigated differences in studies examining the influence of memory rehabilitation interventions for individuals who suffered brain damage as a result of TBI or stroke. One goal of this meta-analysis was to supplement the work published by Cicerone et al. [10, 17] and Rohling et al. [8]. The Cicerone et al. [10, 17] study did not quantitatively evaluate ESs; the Rohling et al. [8] study had a relatively small sample size for memory studies. The present study also examined more covariates than the Rohling et al. [8] meta-analysis. The results obtained in this investigation for the intervention group show a significant moderate effect (r ¼ 0.51) of the interventions which could not be attributed to the passage of time. However, the control analyses show a smaller, but also significant and moderate effect (r ¼ 0.31) for recovery without a treatment intervention. This pattern of results replicated the ES relationship found in Rohling et al. [8]. In both studies, there was a small ES that could not be the result of spontaneous natural recovery, which suggested that the interventions used in the examined studies were responsible for the treatment effect. Unlike Rohling et al. [8], the present analysis found that the difference in ESs between control and intervention conditions was significant. This shows that, overall, the memory rehabilitation strategies produced the desired effect. The confidence intervals computed around the ESs in the control groups indicated that the average ES was significantly greater than zero; this suggests that significant spontaneous improvement in
Efficacy of memory rehabilitation therapy
Brain Inj Downloaded from informahealthcare.com by Flinders University of South Australia on 12/27/14 For personal use only.
DOI: 10.3109/02699052.2014.934921
memory occurs after brain injury and stroke. The fact that some amount of the ES in the experimental treatment condition cannot be attributed to the passage of time suggests that therapeutic intervention may accelerate this improvement. The heterogeneity of ESs in these studies could be accounted for partially by a few moderating variables. Analyses show that studies of stroke recovery produced significantly larger ESs than were apparent in studies of TBI recovery or mixed injury recovery. This ES difference occurred across intervention and control group studies, which suggests that stroke patients experienced more change in their functional status over time and were more responsive to treatment. This finding is inconsistent with published research [25, 26]. Persons with TBI are typically younger than stroke patients and, thus, typically have better prognosis for recovery than stroke patients. Individuals with TBI are more frequently discharged from in-patient facilities and make larger gains in treatment [27]. Their actual recovery, however, may last a lifetime [28, 29]. The larger ESs for stroke studies reported here may be directly related to the particular behavioural and medical treatment approaches implemented with the stroke samples in the studies examined; for example, the use of pharmaceutical treatments [16] to stop haemorrhaging quickly and preventative measures that decreased the level of brain damage when the person had the stroke [27]. Studies addressing working memory produced significantly larger effect sizes relative to studies that involved rehabilitation of other types of memory problems. These results suggest that cognitive rehabilitation therapy may be most effective when applied to stroke rehabilitation and when the goal of the therapy is to improve working memory functions. The fact that significant improvement in memory occurred in the control condition suggests the need for baseline evaluations immediately for individuals who suffered brain damage as a result of TBI or stroke. The results are also interesting regarding which covariates did not predict the ESs. The Beggs’s test suggests that ESs in the selected studies may be over-estimated. Other characteristics of the experimental designs, for example, the number of dependent variables that were measured during the experiment, did not seem to affect the ESs. Surprisingly, 95% confidence intervals computed around ESs for average age of participants, gender, year of publication and whether or not the treatment was computerized showed that none of these variables predicted memory recovery, without a rehabilitation measure. Despite their presence in every study examined, the covariate analyses showed that these factors did not result in statistically significant differences across studies. This does not support findings suggesting that stroke and TBI aetiology varies by age [1]. In addition, the analysis does not support the claim some studies have made suggesting that newer computerized or internet-based rehabilitation strategies demonstrate better overall efficacy relative to traditional methods [14, 15]. Finally, there were no significant differences between the average ES for the treatment and control conditions in the current analysis relative to those reported by Rohling et al. [8], which suggests that the findings reported in both of these meta-analyses are stable.
1615
The present study investigated only memory. It is, therefore, possible that additional meta-analyses of attention, visual spatial and language also may reveal efficacy in those domains as well. Limitations of the current study are primarily due to a lack of available information in the published studies examined. For example, it was generally impossible to describe exactly what the authors had provided relative to treatment intervention. Furthermore, there was generally no discussion of the locus of the neurological insult in any of the studies. The average level of education for the participants was seldom discussed. The present paper suggests that the ESs in studies of stroke survivors are larger than those obtained in studies of TBI. Future studies should provide more rigorous statistical investigations as to why cognitive rehabilitation of memory produces a larger effect in stroke patients. For instance, neuroimaging of brain areas associated with memory in TBI and stroke patients should be documented before and after treatment, as should a standardized qualitative account of the injury profile. Most of the studies examined provided vague descriptions and classifications of stroke or TBI damage in their sample. Many times the only information about participants’ TBI variability was a GCS range. Even this information was impossible to code across studies, as some investigators did not use GCS to define severity and some used it in combination with other variables like neuroimaging and post-traumatic amnesia. Demographic data such as hemispheric location of injury and cognitive deficits of participants were rarely available. The results from the present analysis demonstrate hopeful statistical findings for individuals recovering from TBI and stroke. The available memory rehabilitation treatments appear to be effective; additionally, to a lesser degree, memory also recovers without rehabilitation over time.
Declaration of interest The authors report no conflicts of interest. The authors alone are responsible for the content and writing of the paper.
References 1. Collins M, Iverson G, Lovell M, McKeag D, Norwig J, Maroon J. On-field predictors of neuropsychological and symptom deficit following sports-related concussion. Clinical Journal of Sport Medicine 2003;13:222–229. 2. Englander J, Hall K, Stimpson T, Chaffin S. Mild traumatic brain injury in an insured population: Subjective complaints and return to employment. Brain Injury 1992;6:161–166. 3. Mar J, Arrospide A, Begiristain J, Larran˜aga I, Elosegui E, Oliva-Moreno J. The impact of acquired brain damage in terms of epidemiology, economics and loss in quality of life. BMC Neurology 2011;11:11–46. 4. Kay T. Neuropsychological treatment of mild traumatic brain injury. Journal of Head Trauma Rehabilitation 2003;8:74–75. 5. McCrea M, Iverson G, McAllister T, Hammeke A, Powell M, Barr W, Kelly J. An integrated review of recovery after mild traumatic brain injury (MTBI): Implications for clinical management. The Clinical Neuropsychologist 2009;23:1368–1390. 6. Wolfe CDA. The impact of stroke. British Medical Bulletin 2000; 56:275–286. 7. Cappa SF, Benke T, Clarke S, Rossi B, Stemmer B, van Heugten C. EFNS guidelines on cognitive rehabilitation: Report of an EFNS taskforce. European Journal Neurology 2005;12:665–680. 8. Rohling M, Faust M, Beverly B, Demakis G. Effectiveness of cognitive rehabilitation following acquired
1616
9. 10. 11.
12.
13.
Brain Inj Downloaded from informahealthcare.com by Flinders University of South Australia on 12/27/14 For personal use only.
14.
15.
16.
17.
18. 19. 20. 21. 22. 23. 24. 25.
26.
27.
28.
29.
M. Elliott & F. Parente
brain injury: A meta-analytic re-examination of Cicerone et al.’s (2000, 2005) systematic reviews. Neuropsychology 2009;23:20–39. Planas AM. Advances in stroke. Translational Medicine 2012;44: 318–319. Cicerone K, Dahlberg C, Kalmar K. Evidence-based cognitive rehabilitation: Recommendations for clinical practise. Archive of Physiological Medical Rehabilitation 2000;81:1596–1615. Bergquist T, Gehl C, Mandrekar J, Lepore S, Hanna S, Osten A, Beaulieu W. The effect of internet-based cognitive rehabilitation in persons with memory impairments after severe traumatic brain injury. Brain Injury 2009;23:790–799. Lundqvist A, Grundstro¨m K, Samuelsson K, Ro¨nnberg J. Computerised training of working memory in a group of patients suffering from acquired brain injury. Brain Injury 2010;24: 1173–1183. Livengood M, Anderson J, Schmitter-Edgecombe M. Assessment of memory self-awareness following traumatic brain injury. Brain Injury 2010;24:598–608. McCauley S, Wilde E, Merkley T, Schnelle K, Bigler E, Hunter J, Vasquez A, Levin H. Patterns of cortical thinning in relation to event-based prospective memory performance three months after moderate to severe traumatic brain injury in children. Developmental Neuropsychology 2010;35:318–332. Kim D, Hung TM, Bae K, Jung J, Lee S, Yoon B, Cheong H, Ko K, Ryu J. Gomisin A improves scopolamine-induced memory impairment in mice. European Journal of Pharmacology 2006;542: 29–135. Dou Z, Man D, Ou H, Zheng J, Tam S. Computerised errorless learning-based memory rehabilitation for Chinese patients with brain injury: A preliminary quasi-experimental clinical design study. Brain Injury 2006;20:219–225. Cicerone K, Dahlberg C, Malec J. Evidence-based cognitive rehabilitation: Updated review of the literature from 1998 through 2002. Archive of Physiological Medical Rehabilitation 2005;86: 1681–1692. Park N, Ingles J. Effectiveness of attention rehabilitation after acquired brain injury: A meta-analysis. Neuropsychology 2001;15: 199–210. Robey R. A meta-analysis of clinical outcomes in the treatment of aphasia. Journal of Speech, Language, and Hearing Research 1998; 41:172–187. Katz I, Ashley J, O’Shanick J, Connors S. 2006. Cognitive rehabilitation: The evidence, funding and case for advocacy in brain injury. McLean, VA: Brain Injury Association of America. Rosenthal R, Dimatteo M. Meta-analysis: Recent developments in quantitative methods for literature reviews. Annual Review of Psychology 2001;52:59–82. Cohen J. 1988. Statistical power analysis for the behavioral sciences. 2nd ed. Hillsdale, NJ: Erlbaum. Cooper H, Hedges L. 1994. The handbook of research synthesis. US: Russel Sage. Higgins J, Thompson S, Deeks J, Altman D. Measuring inconsistency in meta-analyses. British Medical Journal 2003;327:557–560. McAllister T, Flashman L, Maerlender A, Greenwald R, Beckwith J, Tosteson T, Crisco J, Brolinson P, Duma S, DuHaime A, et al. Cognitive effects of one season of head impacts in a cohort of collegiate contact sport athletes. Neurology 2012;78:1777–1784. Kraus M, Susmaras T, Caughlin B, Walker C, Sweeney J, Little D. White matter integrity and cognitive in chronic traumatic brain injury: A diffusion tensor imaging study. Brain 2007;130: 2508–2519. Smania N, Avestani R, Roncari L, Ianes P, Girardi P, Varalta V, Gambini MG, Fiaschi A, Gandolfi M. Factors predicting functional and cognitive recovery following severe traumatic, anoxic, and cerebrovascular brain damage. Journal of Head Trauma Rehabilitation 2011;15:45. Teassel R, Bayona N, Marshall S, Cullen N, Bayley M, Chundamala J, Villamere J, Mackie D, Rees L, Hartridge C, et al. A systematic review of rehabilitation of moderate to severe acquired brain injuries. Brain Injury 2007;21:107–112. Gray DS, Burnhanm R. Preliminary outcome analysis of a long-term rehabilitation program for severe acquired brain injury.
Brain Inj, 2014; 28(12): 1610–1616
30.
31. 32.
33.
34.
35.
36. 37.
38.
39.
40.
41. 42.
43. 44.
45.
46.
47.
48.
49.
50.
51.
Archives of Physical Medicine and Rehabilitation 2000;81: 1447–1145. Anderson J, Schmitter-Edgecomb M. Predictions of episodic memory following moderate to severe traumatic brain injury during inpatient rehabilitation. Journal of Clinical Experimental Neuropsychology 2009;31:425–438. Anderson V, Catroppa C. Memory outcome at 5 years postchildhood traumatic brain injury. Brain Injury 2007;21:1399–1409. Severity of Brain Injury [Internet]. Brain Injury Association of America. Available online at: http://biausa.fyrian.com/about-braininjury.htm#causes, accessed 19 September 2012. Berg I, Konning-Haanstra M, Deelman B. Long-term effects of memory rehabilitation. A controlled study. Neuropsychological Rehabilitation 1991;1:97–111. Bourgeois M, Lenius K, Turkstra L, Camp C. The effects of cognitive teletherapy on reported everyday memory behaviours of persons with chronic traumatic brain injury. Brain Injury 2007;21: 1245–1257. Couillet J, Soury S, Lebornec G, Asloun S, Joseph P, Mazaux J, Azouvi P. Rehabilitation of divided attention after severe traumatic brain injury: A randomized trial. Neuropsychological Rehabilitation 2010;20:321–339. Evans J, Wilson B. A memory group for individuals with brain injury. Clinical Rehabilitation 1996;6:75–81. Freeman M, Mittenberg W, DiCowden M, Bat-Americani M. Executive and compensatory memory retraining in traumatic brain injury. Brain Injury 1992;6:65–70. Goldstein G, Beers S, Longmore S, McCue M. Efficacy of memory training: A technological extension and replication. The Clinical Neuropsychologist 1996;10:66–72. Goldstein G, McCue M, Turner S. An efficacy study of memory training for patients with closed head injury. The Clinical Neuropsychologist 1998;2:251–259. Hildebrandt H, Gehrmann A, Modden C, Eling P. Enhancing memory performance after organic brain disease relies on retrieval processes rather than encoding or consolidation. Journal of Clinical and Experimental Neuropsychology 2011;33:257–270. Kerner M, Acker M. Computer delivery of memory retraining with head injured patients. Cognitive Rehabilitation 1985;11:26–31. Krause M, Kennedy M. Metamemory adjustments over time in adults with and without traumatic brain injury. Brain Injury 2009; 23:965–972. Milders M, Deelman B, Berg I. Rehabilitation of memory for people’s names. Memory 1998;6:21–36. Nadar M, McDowd J. ‘Show me, don’t tell me’; is this a good approach for rehabilitation? Clinical Rehabilitation 2008;28: 847–855. Newsome M, Steinberg J, Scheibel R, Troyanskaya M, Chu Z, Hanten G, Levin H. Effects of traumatic brain injury on working memory-related brain activation in adolescents. Neuropsychology 2008;22:419–425. Schacter D, Rich S, Stampp M. Remediation of memory disorders, experimental evaluation of the spaced retrieval technique. Journal of Clinical and Experimental Neuropsychology 1985;7:79–96. Schmitter-Edgecombe M, Fahy J, Whelan J, Long C. Memory remediation after severe closed head injury: Notebook training versus supportive therapy. Journal of Consulting and Clinical Psychology 1995;63:484–489. Serino A, Ciaramelli E, Di Santantonio A, Malagu S, Servadei F, Ladavas E. A pilot study for rehabilitation of central executive deficits after traumatic brain injury. Brain Injury 2007;21:11–19. Stringer A. Ecologically-oriented neurorehabilitation of memory; robustness of outcome across diagnosis and severity. Brain injury 2011;25:169–178. Westerberg H, Jacobaeus H, Hirvikoski T, Clevberger P, Ostensson M, Bartfai A, Klingberg T. Computerised working memory training after stroke- a pilot study. Brain Injury 2007;21: 21–29. Wilson B, Evans J, Emslic H, Malinek V. Evaluation of NeuroPage: A new memory aid. Journal of Neurology, Neurosurgery & Psychiatry 1997;63:113–115.
ORIGIANL ARTICLE
Efficacy of memory rehabilitation therapy: A meta-analysis of TBI and stroke cognitive rehabilitation literature Madison Elliott & Frederick Parente
Brain Inj Downloaded from informahealthcare.com by Flinders University of South Australia on 12/27/14 For personal use only.
Towson University, Towson, MD, USA
Abstract
Keywords
Objective: To examine the efficacy of cognitive rehabilitation strategies specifically designed to improve memory after traumatic brain injury (TBI) and stroke vs. memory improvement with the passage of time. Design and methods: A meta-analysis was performed on 26 studies of memory retraining and recovery that were published between the years of 1985 and 2013. Effect sizes (ESs) from each study were calculated and converted to Pearson’s r and then analysed to assess the overall effect size and the relationship among the ESs, patient demographics and treatment interventions. Results: Results indicated a significant average ES (r ¼ 0.51) in the treatment intervention conditions, as well as a significant average ES (r ¼ 0.31) in the control conditions, in which participants did not receive any treatment. The largest ESs occurred in studies of stroke patients and studies concerning working memory rehabilitation. Conclusions: Results showed that memory rehabilitation was an effective therapeutic intervention, especially for stroke patients and for working memory as a treatment domain. However, the results also indicated that significant memory improvement occurred spontaneously over time.
Cognitive rehabilitation, memory, meta-analysis, neurorehabilitation, remediation, stroke, TBI, working memory
Introduction to brain injury At least 5.3 million US citizens live with a permanent TBI- or stroke-related disability. Health professionals consider TBI to be one of the most debilitating injuries relative to other neurological disorders [1, 2]. Mar et al. [3] detailed the devastating societal and individual impact of neurological trauma. Many factors contribute to a person’s recovery trajectory and, in the past decade, rehabilitation of cognitive processes has become significant in the treatment course [4, 5]. Although cognitive rehabilitation is increasingly popular, clinicians currently face major difficulties when identifying and treating cognitive impairments caused by brain injury or stroke. This is because brain-injured populations typically are a heterogeneous group with a multitude of deficits [6]. Evidence-based treatment guidelines are only as good as the research they rely on and published rehabilitation studies tend to be both elaborate and ambiguous in nature. Several professional committees, including members of the National Institutes of Health (NIH) and the Brain InjuryInterdisciplinary Special Interest Group (BI-ISIG) of the American Congress of Rehabilitation Medicine, have
Correspondence: Madison Elliott, Psychology Department, Towson University, 8000 York Road, Towson, MD 21252, USA. E-mail: [email protected]
History Received 15 May 2013 Revised 14 March 2014 Accepted 6 June 2014 Published online 21 July 2014
attempted to amalgamate this literature via published rehabilitation practice guidelines for TBI and stroke. These guidelines are based on findings from recently published systematic reviews, which generally examine a wide body of published studies that include diverse research designs [7]. Systematic reviews construct valuable inferences about the efficacy of rehabilitation strategies; but what they achieve in variety, they also lack in statistical rigour. Meta-analyses must exclude studies that do not provide group data for effect size (ES) and, therefore, cannot examine as many publications as a systematic review. Nevertheless, meta-analyses are becoming increasingly popular in review methodology in psychological and brain sciences and are recognized as a statistically reliable manner of testing objective observations across studies [8]. The purpose of the present investigation is to provide a meta-analytic review of cognitive rehabilitation strategies for the domain of memory in individuals with TBI and stroke. The authors’ goal is to contribute to future guidelines, as well as augment existing qualitative review findings from memory rehabilitation literature. Cognitive rehabilitation of memory Cognitive Rehabilitation Therapy (CRT) for brain injury and stroke is diverse, with numerous treatment and theoretical models. Treatments can be process-specific, aimed at improving overall performance of a given activity, skill-based,
Brain Inj Downloaded from informahealthcare.com by Flinders University of South Australia on 12/27/14 For personal use only.
DOI: 10.3109/02699052.2014.934921
motor-based or restricted to a certain area of cognition like attention, language, executive function or memory. Overall goals of treatment and treatment implementation are as varied as the treatments themselves [9]. This rehabilitation is broad because of the unique nature of every brain injury. In the case of memory impairment, long-term, short-term and/or working memory can be negatively affected. Rehabilitation tasks and their outcome measures are often developed to improve or evaluate function within a specific type of memory [10]. Memory impairment is commonly reported after brain injury or stroke and individuals who experience trauma localized in the temporal lobes, hippocampus and amygdala are particularly susceptible to memory impairment [10]. However, the locus of the brain injury is not the sole determinant of memory problems; many factors contribute to each person’s unique outcome post-brain injury [2]. To explain the varying nature of TBI, Kay [4] and McCrea et al. [5] have proposed integrated, multi-factorial neuropsychological models of brain injury outcome, including the determination of factors such as neurological health, psychological health, physiological health, as well as their interactions with individual subjective or objective cognitive functions. Various behavioural treatment strategies for memory rehabilitation have proven to be efficacious, such as the internet-based tasks used in Bergquist et al. [11]. Other forms of computerized training [12], awareness questionnaires [13] and motive or incentive-based tasks [14] have also resulted in effective treatment. However, relatively few articles have been published that have documented successful pharmaceutical treatments for memory enhancement [15]. The typical behavioural rehabilitation efficacy study evaluates memory differences for a control group vs. experimental group, pre- and post-intervention design. Outcome measures vary greatly and include neuropsychological batteries, motor-function tests, brain imaging and various self-report instruments. Observer versions of the self-report measures have been administered to participants’ family members, professional supervisors or close friends to obtain a more diverse understanding of their daily behavioural functioning [2]. Although there are published reports of potentially effective treatments, there is no conclusive evidence of an effective standardized treatment programme for memory dysfunction. Most treatments produce improvement in memory functioning, but it is unclear whether some treatments are more efficacious than others [16]. Furthermore, there is little research to document enduring improvement over time for any treatment modality. There is some evidence, however, for treatment efficacy that exceeds the level of improvement that occurs with the passage of time. In Rohling et al.’s [8] meta-analytic re-examination of Cicerone et al.’s [10, 17] review, the authors found a larger effect size for memory improvement in TBI and stroke patients after clinical rehabilitation than in TBI and stroke patients who received no treatment. Memory rehabilitation efficacy CRT after TBI and stroke has been the topic of a number of major reviews evaluating treatment effectiveness [8, 10,
Efficacy of memory rehabilitation therapy
1611
17–19]. In their extensive review, Cicerone et al. [10, 17] evaluated 118 studies that used CRT, classifying them first in terms of methodological quality and then summarizing consistency of findings. The highest ranked studies were those including randomized control trials (RCTs) (I), followed by studies including non-randomized control prospective or retrospective case-controls (II). The lowest ranked studies were those not including control groups or those that used data from case studies of single subjects (III). The study also categorized the research into various domains: attention, vision, perception, apraxia, language and communication, working memory, executive functioning, problem-solving and awareness and comprehensive holistic cognitive rehabilitation. Some of the cognitive rehabilitation domains, such as attention training, visual-spatial training and languagebased interventions produced larger treatment effects than others [17, 20]. More recently, Rohling et al. [8] performed a quantitative meta-analysis of a select, well-controlled, literature set that derived from Cicerone et al. [10, 17]. Rohling et al. [8] concluded there was ‘modest qualitative support’ for CRT efficacy, as asserted by Cicerone et al. [10, 17] asserted in their original paper. Rohling et al. [8] found sufficient evidence that attention training after TBI, language and visual-spatial training for symptoms of stroke were effective. The authors concluded that, out of five major treatmenteffectiveness findings in Cicerone et al. [10, 17], their metaanalysis results only supported three: attention-training, language-based training and visual-spatial training. Rohling et al. [8] cautioned readers that Cicerone et al. [10, 17] did not adequately estimate re-test effects from studies with uncontrolled designs. One of the two major claims Rohling et al. [8] questioned was the efficacy of memory rehabilitation. Although their data showed a moderate effect size (ES) for memory rehabilitation, it was not significant. This suggested that general neuroplasticity and natural recovery were responsible for much of the documented memory recovery, even in the treatment conditions. Due to the lack of significant ESs, the authors also could not definitively say that the memory rehabilitation had the desired effect. However, very few of the 115 articles evaluated in Rohling et al. [8] involved memory rehabilitation, so it is difficult to derive any conclusive evidence about the overall efficacy of memory therapy. The present meta-analysis reviewed studies published in 1985–2013 about memory rehabilitation in participants who suffered a TBI or stroke, in order to build upon the aforementioned findings by Cicerone et al. [10, 17] and Rohling et al. [8], regarding the domain of memory rehabilitation. The primary goal of the present meta-analysis was to update Rohling et al.’s [8] work by adding new studies. A secondary goal was to investigate new factors that may contribute to the recovery effects. Many neuro-rehabilitation techniques have changed and advanced since the Rohling et al. [8] meta-analysis. However, the Rohling et al. [8] methodology for meta-analytic research in this area is exemplary. Therefore, the methods for the current study were adapted from Rohling et al. [8]. These procedures include adding the most recent publications, using the most rigorous and careful methods of the four major
1612
M. Elliott & F. Parente
reviews listed and replicating the Rohling et al. [8] selection criteria. A re-examination of new memory rehabilitation studies, and those in Rohling et al.’s [8] database, revealed several new variables that were significant factors during a patients’ recovery. The following variables were identified based on their prevalence and statistical availability: average age of the patients (paediatric vs. adult), year of publication, type of brain injury, type of memory targeted by the rehabilitation strategy, type of intervention (computerized vs. not-computerized) and sample size.
Brain Inj Downloaded from informahealthcare.com by Flinders University of South Australia on 12/27/14 For personal use only.
Meta-analysis and effect size measures A meta-analytic review is a manner of statistically aggregating and evaluating previously conducted studies. Generally, the purpose of a meta-analysis is to examine the overall effect of a specific variable in available literature. Meta-analyses use research studies as a unit of measure to evaluate central tendency and variability of the ESs across the sample of chosen studies. Rohling et al. [8] used a widely accepted procedure extracting ESs from published studies; the ESs measure of the amount of variance the effect controls in a given experiment. Their meta-analysis was a quantitative evaluation of the extent to which covariates of interest control significant portions of the ESs in the study sample. According to the standards used by Rohling et al.’s [8], the overall effect of the variable is considered reliable if the average ES is significantly different from zero. The goal of the present study was to follow Rohling et al.’s [8] example and to reduce the relevant statistical information in each published study to standard units of ES and then evaluate whether the ESs co-vary with any of several other variables that describe the conditions of the original experiments.
Method Sample of studies Studies were identified using combinations of the search terms: ‘working memory’, ‘rehabilitation’, ‘remediation’, ‘memory’, ‘training’, ‘brain injury’, ‘TBI’ and ‘stroke’ in the following research databases: Medline, Pub Med, PsycINFO, PsycArticles and Google Scholar. Once pertinent articles were identified through these databases, their reference lists were reviewed to locate other potentially relevant studies for inclusion. Based on Rohling et al.’s [8] criteria, excluded studies (1) did not contain a memory-rehabilitative intervention; (2) merely described treatment approaches or theories; (3) were review articles; (4) presented unspecified or unmeasured interventions; (5) lacked a diagnosis or assessment of TBI or stroke; (6) were case studies of a single participant with no empirical data; (7) were non-peer reviewed articles; (8) included exclusively-pharmacological interventions; and (9) were not written in English. Rohling et al. [8] mention that single papers may include multiple analysable studies, with ‘unique non-overlapping samples of participants’ (p. 22). In total, 26 published memory rehabilitation studies met selection standards and could be used for the analysis. Fourteen of the studies compared a treatment intervention condition to a control group.
Brain Inj, 2014; 28(12): 1610–1616
The other 12 studies contained a treatment intervention, but no control condition. Studies not cited in article, but included in the meta-analysis are listed [30–51]. Procedural analysis Meta-analytic procedures using the MIX 2.0 for Microsoft Excel meta-analysis software package were used to evaluate the ESs computed from the available statistics in each published study. These ESs (calculated as the Pearson’s r) were subsequently analysed to determine whether the average ESs were significantly different from zero. The researchers chose to use r because of several advantages listed in Rosenthal and Dimatteo [21]. These authors stated that converting to r is advantageous because this ES, in its point biserial form, better represents the relationship between levels of the independent variable and dependent variable scores. Additionally, converting a continuous r to dichotomous d results in a loss of statistical information. Researchers may use degrees of freedom contrast techniques to analyse trends across a greater number of groups with r and it requires no computational adjustments when examining different types of t-test samples. There is also a well-accepted public familiarity with r and its practical importance is established in the field of psychology [21]. Magnitude of ES (small, moderate, large) was interpreted according to the guidelines published in Cohen [22]. ESs from control conditions estimated how much memory improvement was not attributable to an experimental intervention, but instead attributable to the passage of time and non-specific neuroplasticity. ESs from intervention conditions were subsequently analysed and used to estimate how much memory improvement could be attributed to a cognitive rehabilitation effort. The final analytic procedure compared the first (control analysis) to the second (intervention analysis) in order to determine whether cognitive rehabilitation interventions yield significantly larger ESs compared to control conditions, where no intervention has taken place. The Q statistic was calculated to evaluate the significance of the overall effect size and to assess the significance of the difference between the treatment ES once the control ES was removed. The Q statistic and forest plots were also calculated initially to assess the homogeneity of the ESs across the various studies. The observed heterogeneity of ESs in both control and intervention group conditions was used to justify the selection and use of a random effects model for the analysis. This model allows for comparisons to be generalized beyond studies in the current sample and studies identical to those in the current sample [21]. A random effects model was considered superior to a fixed effects model for analysing the current set of studies because fixed effect models are used when the sample of studies are considered homogenous and of the same population [23]. Although MIX 2.0 analytic procedures weighted ESs by their sample size, Begg’s Test statistics also were calculated to assess whether or not sample sizes affected the reported ESs. The degree of dissemination bias, i.e. the extent to which the published studies over-estimate the size of the effect, was also examined using a funnel plot. Covariates of the ESs, including the average age of the patients (paediatric vs. adult), ratio
DOI: 10.3109/02699052.2014.934921
of males to females in the sample, year of publication, type of brain injury, type of memory targeted by the rehabilitation strategy, type of intervention (computerized vs. notcomputerized) and sample size were also evaluated to determine whether or not they predicted memory recovery in either group.
Results
Brain Inj Downloaded from informahealthcare.com by Flinders University of South Australia on 12/27/14 For personal use only.
Control conditions analysis The average effect size for this condition of r ¼ 0.31 was significant (Z ¼ 10.00, p50.05), which indicated a moderate and significant improvement in memory, that was not attributable to a rehabilitation intervention. Figure 1 is a forest plot displaying the ESs for all included control studies. The plot displays the ES and 95% confidence interval for each included study. The extent to which the horizontal lines overlap is an indication of the homogeneity of the ESs across studies. This visual representation of the data demonstrates that the studies were generally not homogenous in their reported ESs, with two possible outliers. The outliers may be due to different methodology or sample characteristics, as suggested by Control Group Q statistic and selectivity funnel plot results. A significant Q statistic (73.259, p ¼ 0.00) also indicates that the studies included are heterogeneous. The I2 statistic estimates the percentage of within-study variance among the studies due to non-random factors. The I2 statistic 82.25% also suggests heterogeneity among the ESs. It is important to note that when k520, as in the present analysis, both the Q and I2 tests should be interpreted with caution [24]. Nevertheless, this statistic suggests that it may be worthwhile to explore characteristics of the various experiments that may
Figure 1. Forest plot of ESs in control group studies.
Efficacy of memory rehabilitation therapy
1613
have contributed to the inconsistency of methods used among the control studies. Begg’s Test statistics were computed to determine whether publication bias was present in the sample of studies used. Smaller samples with fewer degrees of freedom often produce inflated ESs and may indicate the existence of unpublished studies with lower ESs. The Begg’s statistics computed on these data indicated the presence of publication bias among the chosen studies. Funnel plots can be used to illustrate dissemination bias. If there is little or no dissemination bias, a funnel plot will show all the dots distributed randomly around the synthesis estimate line and they will also fall between the confidence interval funnel lines on each side of the plot. The funnel plot from the present analysis, as seen in Figure 2, indicates the presence of dissemination bias in control condition studies. This also means that there is a possibility of unpublished studies with lower ESs existing and reinforces the Begg’s test results that published studies contain inflated ESs which are possibly over-estimating the effect of recovery without an intervention condition. Intervention conditions analysis The average effect size for this group of studies was r ¼ 0.51 (Z ¼ 20.62, p50.05), which was significantly different from zero. The confidence interval for the overall ES (0.462–0.558), did not include the average ES for the control studies (r ¼ 0.31). The results, therefore, indicated that, overall, a moderate and significant improvement in memory was apparent in the cognitive measures used in these studies. This change in memory function cannot be attributed solely to the passage of time. Figure 3 is a forest plot for the intervention condition studies. The visual representation of the data here demonstrates intervention studies were somewhat less heterogeneous than the controls in their reported ESs. These outliers may be due to different methodologies or publication bias, as suggested by the Intervention Group Q statistic and selectivity funnel plot results. The significant Q statistic indicated significant heterogeneity of the ESs (Q ¼ 81.50, p50.05). The I2 statistic of
Figure 2. Funnel plot for control group studies.
Brain Inj Downloaded from informahealthcare.com by Flinders University of South Australia on 12/27/14 For personal use only.
1614
M. Elliott & F. Parente
Figure 3. Forest plot of ESs in intervention group studies.
Brain Inj, 2014; 28(12): 1610–1616
group conditions indicated that studies of stroke rehabilitation produced significantly larger ESs compared to mixed brain injury and TBI studies (p50.05). Confidence intervals were also used to assess ES differences between the intervention (experimental group) and the control group in studies that included both. The discrepancy between these confidence intervals indicated a significant difference between the experimental and the control group ESs (p50.05). A third confidence interval analysis indicated that studies of working memory produced significantly higher ESs relative to studies of other memory processes. There was no significant difference in ESs for intervention studies (experimental groups) that contained a control group vs. intervention studies that did not contain a control group, for average age of the patients in control and intervention studies (paediatric vs. adult) or type of intervention (computerized vs. not-computerized). There was no significant correlation between ES and the ratio of males-to-females in the sample or for the year of publication in control and intervention studies. Post-hoc correlation analyses investigating the relationship between dependent variables in the control and intervention conditions and ESs did not reveal any significant relationship (p40.05) between the ESs and the number of dependent variables in either the control or the experimental conditions. Additional post-hoc correlation analyses show that there was no significant correlation (p40.05) between sample sizes and ESs for intervention (experiment) group studies, but that there was a significant correlation (p50.05) between sample sizes and ESs for control group studies. There were no other significant covariate/ES correlations.
Discussion
Figure 4. Funnel plot for intervention studies.
69.31% indicates that there was somewhat less heterogeneity than the control conditions where I2 ¼ 82.25%. These statistics should be considered more reliable for the intervention analysis (k ¼ 26) according to sample size criteria in Higgins et al. [24]. The Begg’s test statistics computed on these data showed that dissemination bias was present in the sample of studies used for the analysis. The intervention analysis funnel plot (Figure 4) is an additional indication of the presence of dissemination bias in intervention studies. This shows that, similar to published control studies, published intervention studies have inflated ESs and may be over-estimating the effect of recovery due to rehabilitation. Two of the study covariates predicted the ESs. The 95% confidence intervals computed on the ESs in the intervention
This meta-analytic review investigated differences in studies examining the influence of memory rehabilitation interventions for individuals who suffered brain damage as a result of TBI or stroke. One goal of this meta-analysis was to supplement the work published by Cicerone et al. [10, 17] and Rohling et al. [8]. The Cicerone et al. [10, 17] study did not quantitatively evaluate ESs; the Rohling et al. [8] study had a relatively small sample size for memory studies. The present study also examined more covariates than the Rohling et al. [8] meta-analysis. The results obtained in this investigation for the intervention group show a significant moderate effect (r ¼ 0.51) of the interventions which could not be attributed to the passage of time. However, the control analyses show a smaller, but also significant and moderate effect (r ¼ 0.31) for recovery without a treatment intervention. This pattern of results replicated the ES relationship found in Rohling et al. [8]. In both studies, there was a small ES that could not be the result of spontaneous natural recovery, which suggested that the interventions used in the examined studies were responsible for the treatment effect. Unlike Rohling et al. [8], the present analysis found that the difference in ESs between control and intervention conditions was significant. This shows that, overall, the memory rehabilitation strategies produced the desired effect. The confidence intervals computed around the ESs in the control groups indicated that the average ES was significantly greater than zero; this suggests that significant spontaneous improvement in
Efficacy of memory rehabilitation therapy
Brain Inj Downloaded from informahealthcare.com by Flinders University of South Australia on 12/27/14 For personal use only.
DOI: 10.3109/02699052.2014.934921
memory occurs after brain injury and stroke. The fact that some amount of the ES in the experimental treatment condition cannot be attributed to the passage of time suggests that therapeutic intervention may accelerate this improvement. The heterogeneity of ESs in these studies could be accounted for partially by a few moderating variables. Analyses show that studies of stroke recovery produced significantly larger ESs than were apparent in studies of TBI recovery or mixed injury recovery. This ES difference occurred across intervention and control group studies, which suggests that stroke patients experienced more change in their functional status over time and were more responsive to treatment. This finding is inconsistent with published research [25, 26]. Persons with TBI are typically younger than stroke patients and, thus, typically have better prognosis for recovery than stroke patients. Individuals with TBI are more frequently discharged from in-patient facilities and make larger gains in treatment [27]. Their actual recovery, however, may last a lifetime [28, 29]. The larger ESs for stroke studies reported here may be directly related to the particular behavioural and medical treatment approaches implemented with the stroke samples in the studies examined; for example, the use of pharmaceutical treatments [16] to stop haemorrhaging quickly and preventative measures that decreased the level of brain damage when the person had the stroke [27]. Studies addressing working memory produced significantly larger effect sizes relative to studies that involved rehabilitation of other types of memory problems. These results suggest that cognitive rehabilitation therapy may be most effective when applied to stroke rehabilitation and when the goal of the therapy is to improve working memory functions. The fact that significant improvement in memory occurred in the control condition suggests the need for baseline evaluations immediately for individuals who suffered brain damage as a result of TBI or stroke. The results are also interesting regarding which covariates did not predict the ESs. The Beggs’s test suggests that ESs in the selected studies may be over-estimated. Other characteristics of the experimental designs, for example, the number of dependent variables that were measured during the experiment, did not seem to affect the ESs. Surprisingly, 95% confidence intervals computed around ESs for average age of participants, gender, year of publication and whether or not the treatment was computerized showed that none of these variables predicted memory recovery, without a rehabilitation measure. Despite their presence in every study examined, the covariate analyses showed that these factors did not result in statistically significant differences across studies. This does not support findings suggesting that stroke and TBI aetiology varies by age [1]. In addition, the analysis does not support the claim some studies have made suggesting that newer computerized or internet-based rehabilitation strategies demonstrate better overall efficacy relative to traditional methods [14, 15]. Finally, there were no significant differences between the average ES for the treatment and control conditions in the current analysis relative to those reported by Rohling et al. [8], which suggests that the findings reported in both of these meta-analyses are stable.
1615
The present study investigated only memory. It is, therefore, possible that additional meta-analyses of attention, visual spatial and language also may reveal efficacy in those domains as well. Limitations of the current study are primarily due to a lack of available information in the published studies examined. For example, it was generally impossible to describe exactly what the authors had provided relative to treatment intervention. Furthermore, there was generally no discussion of the locus of the neurological insult in any of the studies. The average level of education for the participants was seldom discussed. The present paper suggests that the ESs in studies of stroke survivors are larger than those obtained in studies of TBI. Future studies should provide more rigorous statistical investigations as to why cognitive rehabilitation of memory produces a larger effect in stroke patients. For instance, neuroimaging of brain areas associated with memory in TBI and stroke patients should be documented before and after treatment, as should a standardized qualitative account of the injury profile. Most of the studies examined provided vague descriptions and classifications of stroke or TBI damage in their sample. Many times the only information about participants’ TBI variability was a GCS range. Even this information was impossible to code across studies, as some investigators did not use GCS to define severity and some used it in combination with other variables like neuroimaging and post-traumatic amnesia. Demographic data such as hemispheric location of injury and cognitive deficits of participants were rarely available. The results from the present analysis demonstrate hopeful statistical findings for individuals recovering from TBI and stroke. The available memory rehabilitation treatments appear to be effective; additionally, to a lesser degree, memory also recovers without rehabilitation over time.
Declaration of interest The authors report no conflicts of interest. The authors alone are responsible for the content and writing of the paper.
References 1. Collins M, Iverson G, Lovell M, McKeag D, Norwig J, Maroon J. On-field predictors of neuropsychological and symptom deficit following sports-related concussion. Clinical Journal of Sport Medicine 2003;13:222–229. 2. Englander J, Hall K, Stimpson T, Chaffin S. Mild traumatic brain injury in an insured population: Subjective complaints and return to employment. Brain Injury 1992;6:161–166. 3. Mar J, Arrospide A, Begiristain J, Larran˜aga I, Elosegui E, Oliva-Moreno J. The impact of acquired brain damage in terms of epidemiology, economics and loss in quality of life. BMC Neurology 2011;11:11–46. 4. Kay T. Neuropsychological treatment of mild traumatic brain injury. Journal of Head Trauma Rehabilitation 2003;8:74–75. 5. McCrea M, Iverson G, McAllister T, Hammeke A, Powell M, Barr W, Kelly J. An integrated review of recovery after mild traumatic brain injury (MTBI): Implications for clinical management. The Clinical Neuropsychologist 2009;23:1368–1390. 6. Wolfe CDA. The impact of stroke. British Medical Bulletin 2000; 56:275–286. 7. Cappa SF, Benke T, Clarke S, Rossi B, Stemmer B, van Heugten C. EFNS guidelines on cognitive rehabilitation: Report of an EFNS taskforce. European Journal Neurology 2005;12:665–680. 8. Rohling M, Faust M, Beverly B, Demakis G. Effectiveness of cognitive rehabilitation following acquired
1616
9. 10. 11.
12.
13.
Brain Inj Downloaded from informahealthcare.com by Flinders University of South Australia on 12/27/14 For personal use only.
14.
15.
16.
17.
18. 19. 20. 21. 22. 23. 24. 25.
26.
27.
28.
29.
M. Elliott & F. Parente
brain injury: A meta-analytic re-examination of Cicerone et al.’s (2000, 2005) systematic reviews. Neuropsychology 2009;23:20–39. Planas AM. Advances in stroke. Translational Medicine 2012;44: 318–319. Cicerone K, Dahlberg C, Kalmar K. Evidence-based cognitive rehabilitation: Recommendations for clinical practise. Archive of Physiological Medical Rehabilitation 2000;81:1596–1615. Bergquist T, Gehl C, Mandrekar J, Lepore S, Hanna S, Osten A, Beaulieu W. The effect of internet-based cognitive rehabilitation in persons with memory impairments after severe traumatic brain injury. Brain Injury 2009;23:790–799. Lundqvist A, Grundstro¨m K, Samuelsson K, Ro¨nnberg J. Computerised training of working memory in a group of patients suffering from acquired brain injury. Brain Injury 2010;24: 1173–1183. Livengood M, Anderson J, Schmitter-Edgecombe M. Assessment of memory self-awareness following traumatic brain injury. Brain Injury 2010;24:598–608. McCauley S, Wilde E, Merkley T, Schnelle K, Bigler E, Hunter J, Vasquez A, Levin H. Patterns of cortical thinning in relation to event-based prospective memory performance three months after moderate to severe traumatic brain injury in children. Developmental Neuropsychology 2010;35:318–332. Kim D, Hung TM, Bae K, Jung J, Lee S, Yoon B, Cheong H, Ko K, Ryu J. Gomisin A improves scopolamine-induced memory impairment in mice. European Journal of Pharmacology 2006;542: 29–135. Dou Z, Man D, Ou H, Zheng J, Tam S. Computerised errorless learning-based memory rehabilitation for Chinese patients with brain injury: A preliminary quasi-experimental clinical design study. Brain Injury 2006;20:219–225. Cicerone K, Dahlberg C, Malec J. Evidence-based cognitive rehabilitation: Updated review of the literature from 1998 through 2002. Archive of Physiological Medical Rehabilitation 2005;86: 1681–1692. Park N, Ingles J. Effectiveness of attention rehabilitation after acquired brain injury: A meta-analysis. Neuropsychology 2001;15: 199–210. Robey R. A meta-analysis of clinical outcomes in the treatment of aphasia. Journal of Speech, Language, and Hearing Research 1998; 41:172–187. Katz I, Ashley J, O’Shanick J, Connors S. 2006. Cognitive rehabilitation: The evidence, funding and case for advocacy in brain injury. McLean, VA: Brain Injury Association of America. Rosenthal R, Dimatteo M. Meta-analysis: Recent developments in quantitative methods for literature reviews. Annual Review of Psychology 2001;52:59–82. Cohen J. 1988. Statistical power analysis for the behavioral sciences. 2nd ed. Hillsdale, NJ: Erlbaum. Cooper H, Hedges L. 1994. The handbook of research synthesis. US: Russel Sage. Higgins J, Thompson S, Deeks J, Altman D. Measuring inconsistency in meta-analyses. British Medical Journal 2003;327:557–560. McAllister T, Flashman L, Maerlender A, Greenwald R, Beckwith J, Tosteson T, Crisco J, Brolinson P, Duma S, DuHaime A, et al. Cognitive effects of one season of head impacts in a cohort of collegiate contact sport athletes. Neurology 2012;78:1777–1784. Kraus M, Susmaras T, Caughlin B, Walker C, Sweeney J, Little D. White matter integrity and cognitive in chronic traumatic brain injury: A diffusion tensor imaging study. Brain 2007;130: 2508–2519. Smania N, Avestani R, Roncari L, Ianes P, Girardi P, Varalta V, Gambini MG, Fiaschi A, Gandolfi M. Factors predicting functional and cognitive recovery following severe traumatic, anoxic, and cerebrovascular brain damage. Journal of Head Trauma Rehabilitation 2011;15:45. Teassel R, Bayona N, Marshall S, Cullen N, Bayley M, Chundamala J, Villamere J, Mackie D, Rees L, Hartridge C, et al. A systematic review of rehabilitation of moderate to severe acquired brain injuries. Brain Injury 2007;21:107–112. Gray DS, Burnhanm R. Preliminary outcome analysis of a long-term rehabilitation program for severe acquired brain injury.
Brain Inj, 2014; 28(12): 1610–1616
30.
31. 32.
33.
34.
35.
36. 37.
38.
39.
40.
41. 42.
43. 44.
45.
46.
47.
48.
49.
50.
51.
Archives of Physical Medicine and Rehabilitation 2000;81: 1447–1145. Anderson J, Schmitter-Edgecomb M. Predictions of episodic memory following moderate to severe traumatic brain injury during inpatient rehabilitation. Journal of Clinical Experimental Neuropsychology 2009;31:425–438. Anderson V, Catroppa C. Memory outcome at 5 years postchildhood traumatic brain injury. Brain Injury 2007;21:1399–1409. Severity of Brain Injury [Internet]. Brain Injury Association of America. Available online at: http://biausa.fyrian.com/about-braininjury.htm#causes, accessed 19 September 2012. Berg I, Konning-Haanstra M, Deelman B. Long-term effects of memory rehabilitation. A controlled study. Neuropsychological Rehabilitation 1991;1:97–111. Bourgeois M, Lenius K, Turkstra L, Camp C. The effects of cognitive teletherapy on reported everyday memory behaviours of persons with chronic traumatic brain injury. Brain Injury 2007;21: 1245–1257. Couillet J, Soury S, Lebornec G, Asloun S, Joseph P, Mazaux J, Azouvi P. Rehabilitation of divided attention after severe traumatic brain injury: A randomized trial. Neuropsychological Rehabilitation 2010;20:321–339. Evans J, Wilson B. A memory group for individuals with brain injury. Clinical Rehabilitation 1996;6:75–81. Freeman M, Mittenberg W, DiCowden M, Bat-Americani M. Executive and compensatory memory retraining in traumatic brain injury. Brain Injury 1992;6:65–70. Goldstein G, Beers S, Longmore S, McCue M. Efficacy of memory training: A technological extension and replication. The Clinical Neuropsychologist 1996;10:66–72. Goldstein G, McCue M, Turner S. An efficacy study of memory training for patients with closed head injury. The Clinical Neuropsychologist 1998;2:251–259. Hildebrandt H, Gehrmann A, Modden C, Eling P. Enhancing memory performance after organic brain disease relies on retrieval processes rather than encoding or consolidation. Journal of Clinical and Experimental Neuropsychology 2011;33:257–270. Kerner M, Acker M. Computer delivery of memory retraining with head injured patients. Cognitive Rehabilitation 1985;11:26–31. Krause M, Kennedy M. Metamemory adjustments over time in adults with and without traumatic brain injury. Brain Injury 2009; 23:965–972. Milders M, Deelman B, Berg I. Rehabilitation of memory for people’s names. Memory 1998;6:21–36. Nadar M, McDowd J. ‘Show me, don’t tell me’; is this a good approach for rehabilitation? Clinical Rehabilitation 2008;28: 847–855. Newsome M, Steinberg J, Scheibel R, Troyanskaya M, Chu Z, Hanten G, Levin H. Effects of traumatic brain injury on working memory-related brain activation in adolescents. Neuropsychology 2008;22:419–425. Schacter D, Rich S, Stampp M. Remediation of memory disorders, experimental evaluation of the spaced retrieval technique. Journal of Clinical and Experimental Neuropsychology 1985;7:79–96. Schmitter-Edgecombe M, Fahy J, Whelan J, Long C. Memory remediation after severe closed head injury: Notebook training versus supportive therapy. Journal of Consulting and Clinical Psychology 1995;63:484–489. Serino A, Ciaramelli E, Di Santantonio A, Malagu S, Servadei F, Ladavas E. A pilot study for rehabilitation of central executive deficits after traumatic brain injury. Brain Injury 2007;21:11–19. Stringer A. Ecologically-oriented neurorehabilitation of memory; robustness of outcome across diagnosis and severity. Brain injury 2011;25:169–178. Westerberg H, Jacobaeus H, Hirvikoski T, Clevberger P, Ostensson M, Bartfai A, Klingberg T. Computerised working memory training after stroke- a pilot study. Brain Injury 2007;21: 21–29. Wilson B, Evans J, Emslic H, Malinek V. Evaluation of NeuroPage: A new memory aid. Journal of Neurology, Neurosurgery & Psychiatry 1997;63:113–115.
