Psychology and Aging 2014, Vol. 29. No. 2, 250-263
© 2014 American Psychological Association 0882-7974/14/$l2.00 DOI: 10.!037/a0035908
Examining the Relationship Between Subjective and Objective Memory Performance in Older Adults: A Meta-Analysis Jessica J. Cramley, Cinnamon A. Stetler, and Michelle Horhota Furman University Are the beliefs that older adults hold about their memory abilities associated wi± their scores on lab-based memory tasks? A review of the aging literature suggests that the correlation between subjective and objective memory is inconsistent, with some studies reporting significant effects and others reporting null results. A meta-analysis was conducted to quantitatively summarize the relationship between subjective memory, defined as general predictions about memory, and objective memory performance in older adults, and to examine the conditions under which this relationship may be strongest. This meta-analysis included 53 studies, each of which included a normatively aging older adult sample. Overall, the association between subjective and objective memory was small (r = .062, SE = 0.014) but reliably greater than zero. Moderator analyses were conducted to better understand the parameters of this effect. Age, years of education, gender, depression symptoms, length and format of subjective memory measures, and type of objective memory were significantly correlated, with effect size. These results caution against relying on general subjective memory belief measures as a substitute for objective assessments of memory. Keywords: memory beliefs, memory complaint, memory performance, older adults, meta-analysis
tively impact an individual's well-being (Derouesné, Lacomblez, Thibault, & LePoncin, 1999; Mol et al., 2007; Verhaeghen, Geraerts, & Marcoen, 2000). Thus, subjective beliefs are an important component of the memory process that needs to be better understood. Although much empirical work has been done in this area, a consensus regarding the relationship between self-reported memory performance and objective memory performance in older adults has not emerged. Although some studies fail to demonstrate an association between subjective and objective memory (Cook & Marsiske, 2006; Derouesné et al, 1999; Mendes et al., 2008), other studies find significant correlations (Van Bergen, Jelicic, & Merckelbach, 2009). One reason for the inconsistent results may be that there actually is no relationship between general subjective beliefs and objective memory performance among older adults. Although memory does decline over time, the decline may be too subtle to self-assess accurately (Schmand, Jonker, Geerlings, & Lindeboom, 1997). Therefore, any attempt to characterize the relationship capitalizes on chance and random error (with some studies reporting true null findings). A second reason may be that the association is real but small. If studies are underpowered to find the effect, or if subjective memory is measured unreliably, it would lead to a high rate of Type II error. The present study uses meta-analytic techniques to determine whether the relationship between general subjective memory beliefs and objective memory is reliably nonzero, and if so, what the magnitude of the effect is. A meta-analysis is appropriate in this case because it provides a systematic review of the primary literature and provides a reliable quantitative synthesis of the available data that does not rely solely on the results of significance testing.
A large body of evidence suggests that Americans tend to hold negative views toward aging, with individuals expecting declines in competence (Kite, Stockdale, Whitely, & Johnson, 2005) and memory performance with age (Lineweaver & Hertzog, 1998). Older adults themselves typically report that although memory performance tends to decline over time, they personally believe that they will be equal to (or slightly better than) most other older adults (Lineweaver & Hertzog, 1998; Ryan & Kwong See, 1993). In actuality, individual differences and the amount of variability in older adults' cognitive abilities is large, (Zacks, Hasher, & Li, 2000) and everyone is most certainly not "better than average." This suggests a disconnect between how older adults perceive their memory performance and how they actually perform. It is important to understand whether older individuals are able to accurately assess their memory performance because individuals who are metacognitively aware are more likely to implement effective strategies, which can subsequently improve memory performance (Lachman & Andreoletti, 2006). Individuals who overor underestimate their memory abilities may not invest the required effort into a task, which can result in poor performance. These beliefs can also influence an individual's self-concept more broadly as they relate to anxiety about memory, which can nega-
Jessica J. Crumley, Cinnamon A. Stetler, and Michelle Horhota, Department of Psychology, Furman University. We thank Douglas Bonett for his advice and assistance with the metaanalytic methods and statistics. Correspondence concerning this article should be addressed to Michelle Horhota, Department of Psychology, Eurman University, 3300 Poinsett Hwy, Greenville, SC 29613. E-mail: [email protected] 250
SUBJECTIVE AND OBJECTIVE MEMORY IN OLDER ADULTS
Demographic Moderators Correlations between subjective memory and objective memory tend to be small and inconsistent. This may be due, in part, to demographic variables that may moderate the size of the effect. For example, age and education may be important to consider. Parisi et al. (2011) suggest that the older the sample, the larger the correlation between subjective and objective memory should be. Oldest-old adults may experience sizable cognitive declines that are readily perceived and are therefore more likely to be reported compared with the subtle objective declines that young-old adults may experience. Furthermore, the correlation may be larger when a study consists of a highly educated sample (Barker, Jones, & Jennison, 1995; Bolla, Lindgren, Bonaccorsy, & Bleeker, 1991; Zelinski, Bumight, & Lane, 2001). Those who are highly educated may have more insight into their own memory abilities (Zelinski et al, 2001), and also may be more likely to be aware of and use memory strategies to help compensate for any perceived decline (Lachman & Lachman, 1980). Thus, we predict that studies that include older or more educated samples will report larger associations between subjective and objective memory compared with studies with younger or less educated samples. Another potential moderator is the presence of depressive symptomology. Over 10% of adults over the age of 60 report suffering from an episode of depression in their lifetime, and many more experience elevated levels of negative affect or other symptoms that do not exceed a clinical threshold (Kessler et al., 2005). Studies conducted with normatively aging older adults have demonstrated that individuals with higher levels of depression tend to report higher levels of subjective memory complaints and lower levels of perceived memory ability (Bolla et al., 1991; Harwood, Barker, Ownby, Mullan, & Duara, 2004). Thus, it may be that elevated levels of depression, even in nonclinical samples, may minimize the relationship between subjective memory judgments and objective memory performance. Demographic variables such as gender and ethnicity have not been traditionally considered to moderate the relationship between subjective and objective memory. However, women and ethnic minorities are more likely to be influenced by variables that are known to moderate the relationship. For example, women and ethnic minorities tend to experience higher levels of depression (Kessler, 2006; Williams et al., 2007) and have lower education levels (Baker & Vêlez, 1996) than their Caucasian male counterparts. Because higher levels of education and lower levels of depression are thought to be associated with larger correlations, studies with larger proportions of women or ethnic minority members in their samples may yield smaller effect sizes than those studies with higher proportions of men or Caucasians.
Variations in Memory Measurements In the cognitive aging literature, subjective memory has been operationalized in a variety of different ways. For example, a distinction is made between "local" measures of subjective memory and "global" measures of subjective memory (Hertzog, Dixon, & Hultsch, 1990). Local measures are judgments that are specific to a context, and can be considered task-specific self-efficacy beliefs. For example, when participants make a prediction about their performance on a specific measure after receiving a brief description of the task, it would be considered a "local" or "task-
251
specific" subjective measure. Alternatively, global measures of subjective memory are more generalized and are removed from specific context (Hertzog et al, 1990). Global measures typically are assessed in the form of standardized questionnaires (e.g.. Memory Functioning Questionnaire [MFQ], Gilewski, Zehnski, & Schaie, 1990; Metamemory in Adulthood Questionnaire, Dixon, Hultsch, & Hertzog, 1988; Memory Assessment Clinics SelfRating Scale, Crook & Larrabee, 1992; Subjective Memory Complaints Questionnaire, Youn et a!., 2009; Cognitive Dysfunction Questionnaire, Vestergren, Ronnlund, Nyberg, & Nilsson, 2011). These measures typically aggregate responses across a range of memory types to create a general abstracted assessment of subjective memory ability. Although some global memory questionnaires include a "memory self-efficacy" factor, that factor tends to refer to general feelings of self-efficacy for memory tasks as a whole, rather than for specific memory tasks individually. Because global and task-specific measures assess different constructs, we focused our review on global performance judgment measures, and did not include studies that used only task-specific measures (see Beaudoin & Desrichards, 2011, for a review of this literature). Another important distinction in the literature is the timing of the subjective memory assessment. Subjective memory judgments can occur before the objective memory measure (a prediction) or after the objective memory measure (a postdiction). Past work has shown that the accuracy of a judgment is enhanced when subjective judgments follow performance of the memory task (Connor, Dunlosky, & Hertzog, 1997; Hertzog et al., 1990). Thus, postdictions refiect metacognitive monitoring accuracy, or how well a person monitors and updates their knowledge of task performance. In contrast, predictions reflect global beliefs in the absence of specific experience with the task (Connor et al, 1997). For the purpose of this meta-analysis, we focused solely on global predictions of performance, made prior to a participants' experience with a task, and excluded studies that focused on knowledge updating and monitoring. Even among global memory measures, important differences exist in the way the items are ñ'amed. Some global memory measures assess information about the participant's current perceived memory ability and functioning (e.g., Metamemory in Adulthood Questionnaire, Dixon et al., 1988; Multifactorial Memory Questionnaire, Troyer & Rich, 2002), whereas others tend to focus specifically on the frequency of memory failures and memory complaints (e.g.. Memory Assessment Clinics Self-Rating Scale; Crook & Larrabee, 1992). Research has suggested that the magnitude of the correlation between subjective and objective memory may vary depending on whether memory ability or memory complaints are used to assess subjective memory, with memory complaints correlating less strongly with memory performance (Balash et al., 2013; Hertzog & Pearman, in press; Zehnski, Gilewski, & Thompson, 1980). Global measures of subjective memory can also vary in their length and level of detail, as well as in format (questionnaire vs. interview). It may be the case that longer assessments more accurately reflect a person's subjective beliefs than shorter assessments, resulting in higher correlations with objective memory performance (Potter & Hartman, 2006; Reid & Maclullich, 2006; Wahlin, Maitland, Backman, & Dixon, 2003). Further, questionnaires tend to be more specific and longer than interviews (Jorm, Christensen, Körten, Jacomb, & Henderson, 2001). Interview questions are also less structured and more open-
CRUMLEY, STETLER, AND HORHOTA
252
ended (e.g., "Do you have any trouble remembering things that have happened recently?"; Jorm et al., 2001), and responses are coded by the interviewer on a scale of severity of complaint. Because questionnaires have more questions than interviews, and also tend to be more structured and less subject to interviewer biases, questionnaires may result in reliably larger correladons (Jungwirth et al., 2004). The way in which objective memory is assessed may also help to determine the magnitude of the effect size. Traditional lab-based memory tasks have been criticized for not being grounded in everyday memory performance (Sunderland, Harris, & Baddeley, 1983). Often, participants are assessed using objective measures with which they have no experience. Therefore, participants may rate their subjective memory based on everyday memory performance while being assessed on unfatniliar laboratory tasks that do not approximate daily life. Several studies suggest that the correlation between subjective and objective memory is higher when the study uses ecologically valid objective memory measures (e.g., shopping list recall or face name recall; Berry, West, & Dennehey, 1989; Schmidt, Berg, & Deelman, 2001; Sunderland, Watts, Baddeley, & Harris, 1986). Similarly, the type of memory assessed through objective measurement may influence the effect size. Individuals of all ages have different beliefs about the different forms of memory (Lineweaver & Hertzog, 1998). If personal memory beliefs vary across memory types, the correlations between beliefs and performance may differ based on memory type. Prospective memory, for example, seems to yield larger subjective and objective correlations compared with retrospective memory or working memory (Zeintl, Kliegel, Rast, & Zimprich, 2006). The reason for this may be that prospecfive memory performance is particularly salient to everyday life. Because of this, it is possible that prospective memory failures may be more easily recognized, and, therefore, older adults may be able to accurately track changes in prospective memory performance compared with episodic or working memory.
Current Study The goals of the current study are to first clarify the nature of any association between subjective and objective memory among normatively aging older adults by using meta-analytic techniques. Second, the study will determine whether sample characteristics, subjective memory measure variations, type of memory in the objective assessment, and level of ecological validity of objective memory measures influence the magnitude of the subjectiveobjective correlation.
Method Literature Search To obtain relevant studies, several search terms were entered into PsycINFO, MEDLINE, and Academic Search Premier databases. The first search field included the term "objective memory" or "memory performance." The second search field included the term "subjective memory" or "memory complaint" or "memory beliefs." The third search field included the term "older adults." In a fourth search field the following questionnaire names were included: "Memory Complaint Questionnaire" or "Memory Abil-
ity Questionnaire" or "Cognitive Failures Questionnaire" or "Metamemory in Adulthood Questionnaire" or "Memory Functioning Questionnaire." Our search covered published articles that were available prior to the end of February 2012. Reference secfions of review articles were consulted to identify relevant studies that were not identified through the computerized search. The authors also attempted to contact researchers in the field in order to obtain any unpublished findings. Studies were not restricted by date of publication or country of origin, but must have been published in English. Studies were included in the abstract screening process if they enrolled participants over 60 years of age. The search yielded 1,737 results whose abstracts were then reviewed for the following inclusion criteria.
Inclusion Criteria In order to be included in the meta-analysis, a study had to report at least one correlation between subjecfive memory, defined as a global prediction of memory performance in advance of completing a memory task, and objective memory. Fifty-seven percent of the studies were excluded because they did not measure both subjective and objective memory. Six percent of the studies were excluded because they did not report the correlation between subjective and objective measures of memory. Some studies included adults who were considered to have mild cognitive impairment (MCI) in their samples. The aim of this meta-analysis was to investigate the relationship between objective and subjective memory in normatively aging older adults; therefore, if these studies reported only one main effect, the proportion of participants within the sample with MCI had to be less than 30% in order to be included. Thiriy-three percent of the search results did not meet this criterion (i.e., included a sample that exceeded the 30% MCI limit). The cutoff of 30% was selected in order to balance the heterogeneity created by including clinical samples with our desire to include a generalizable sample of studies that was not overly narrow. If a study included over 30% MCI participants, but reported the correlation between subjective and objective memory for the non-MCI portion of their sample separately, only the effect size for the non-MCI sample was used. The final number of studies included in the current meta-analysis was 53. All studies are reported in Table 1 with their respective coding classifications. All coding was done by the first author. She consulted with the other authors in rare cases in which ambiguity made coding unclear, and coding was then based on consensus. Coding for IVIoderators Sample characteristics. The information that was extracted about each sample included age, gender composition (percent female), education level (in years), race/ethnic composition (percent Caucasian), mean depression score, and whether the study screened participants for MCI. The name of the depression measure that each study used was also recorded. In order to provide meaningful comparisons across different depression scales, depression means were standardized among the studies that used the same symptom measure. Because the presence of cogtiitive impairments could influence the association between objective and subjective memory, we coded each study on whether they screened and excluded people with MCI from analysis. A study was not
SUBJECTIVE AND OBJECTIVE MEMORY IN OLDER ADULTS included in this category if it was said to be testing normatively aging older adults but did not mention any type of screening procedure. This allowed us to determine whether samples known to be without cognitive impairment yield different effects compared with studies that only assume their samples are without cognitive impairment. Memory assessment. Subjective memory. The method used to assess global subjective memory was coded as being either an interview or questionnaire. To examine the effect of assessment length, assessments with 10 or fewer items were coded as "short." Assessments with more than 10 items were coded as "long." When possible, the subjective memory measures were categorized based on whether they assessed memory complaints, memory capacity, or both. A memory complaint directly yields a measure of subjective memory failures (e.g., "How often do names present a problem for you?"; Memory Functioning Questionnaire, Gilewski et al., 1990), whereas memory capacity captures one's perceived memory abilities (e.g., "I can remember the things I need to"; Memory Controllability Inventory, Lachman, Bandura, Weaver, & Elliott, 1995). Objective memory. Objective memory was coded based on the type of tnemory that was assessed: short-term, long-term, working, prospective, episodic (including paired-associates tasks), visual, semantic, and implicit memory as claimed by the author of a particular measure. Each measure was coded into only one category. The ecological validity of the objective memory measures was classified on the percentage of the objective tasks that approximated everyday memory performance. Tasks that were classified as everyday objective memory tasks included short story recall, face-name paired associates, facial recognition, shopping list recall, everyday object placement recall, noncomputerized prospective memory tasks (e.g., the blue card test, the red pencil task, call the lab tasks), photograph detail recall, celebrity name recall, lists of daily tasks recall (e.g., lists of appointments and errands), map/direction recall, doctor instructions recall, and personal recent and remote events recall. If a study's objective task battery contained one third or fewer everyday tasks, it was classified as a "low." A study was classified as a "moderate" if one third to two thirds of the objective memory tasks were ecologically valid, and as a "high" if more than two thirds of the objective tasks were ecologically valid.
Statistical Analysis The Pearson r correlation(s) between subjective and objective memory served as the dependent variable in all analyses (Rosenthal & DiMatteo, 2001). It is important for each smdy to contribute only one effect in order to meet the assumption of independent observations. If multiple correlations were reported within one study, the correlations were averaged to represent that study's overall effect. However, multiple effects within a study were retained when applicable and used in the relevant moderator analyses. Twenty-three of the 53 studies reported multiple effects, for a total of 87 unique effects. The overall mean effect (unweighted), standard error, and 95% confidence interval were computed under a varying coefficient (VC) model (Bonett, 2008). The VC model is an appropriate model because it does not assume homogeneity among effect sizes across studies (as a fixed effects
253
model does), nor does it assume the studies included in the meta-analysis are a random sample from some definable and normally distributed population of studies (as a random effects model does). Effects can be interpreted as significantly different from zero when the associated confidence interval does not include zero. All effect size, variance, and moderator calculations were completed with formulas from Bonett (2008) using SAS 9.2, except for the analysis of objective memory type, in whicb several studies reported multiple effects per study. In order to avoid the loss of data that would occur if those effects had been simply averaged, we employed hierarchical linear modeling techniques to account for the dependence in these data. Analyses involving this variable (objective memory type) were conducted using HLM 7.0 software.
Results Fifty-three studies representing 20,319 total participants were included in the meta-analysis. Information on each study can be found in Table 1. Publication dates ranged from 1975 to 2011 (mode = 2008). The average sample size was 383.38 {SD = 963.50, Mdn = 114, range = 20 to 6,772). Across all 53 smdies, correlations (Pearson r) between subjective and objective memory ranged from -0.290 to 0.410. Under a VC model (Bonett, 2008), the unweighted mean effect size was r = .062 {SE = 0.014, 95% CI [0.035, .090]). Although reliably greater than zero, the average correlation is very small in magnitude (Cohen, 1988). Subjective memory assessment explained less than 1% of the variance in objective memory performance. With an effect this small, many studies may not have found a significant effect and thus may have been less likely to be published. In order to determine whether the reported mean effect size could have been influenced by publication bias, we conducted three tests. First, we computed the number of unpublished studies with an effect size below 0.035 that would have to exist in order to reduce the average correlation below the level of significance. Using Orwin's (1983) fail-safe N formula, approximately 44 additional unpublished studies with null findings would have to exist. • Given that this number is approximately 83% the size of the current sample of studies, we believe that our reported mean effect size is unlikely to be only a product of publication bias. Second, we created a funnel plot (see Figure 1) to visually examine whether asymmetry due to publication bias was present. A visual inspection of the funnel plot does not reveal the typical pattern of asymmetry that would suggest the presence of publication bias. Twenty-nine of the effects are located to the left of the mean, and 24 of the effects are to the right of the mean. As a third measure of publication bias, we examined the association between estimates of effect size and their variances, as described by Begg and Mazumdar (1994); by this formal test, the current data set does not reveal publication bias, r = —0.032, p — .82. There is much debate in the literature surrounding the construction and interpretation of funnel plots (Tang & Liu, 2000; Terrin, Schmid, & Lau, 2005). The pattern of effects in Figure 1 is atypical in that the distribution does not resemble a funnel-like shape. Typically, a funnel shape emerges because larger, more precise studies, that is, those with small standard errors, cluster more tightly around the average effect size, whereas smaller, less precise studies yield more variable results. Instead, in our data set, studies
CRUMLEY, STETLER, AND HORHOTA
254 OO
CO 8 oo O
CO CO
IT)
(N
d d1
1
1
t'S r- es
d
Ö
d
d
o I
p
rs
(N es
S3
SS
Ö
II II
in SD
m' m'
o°
1^ Uu
O
O
SÖ
II es
^
o
CO
O
s
—
'S u
—.
(NI
(N —
_
_
_-
(-sj
-- —
(N
_
.-.
^
es
(D
d
81.63 63.64
64.7 80.23
72.84 63.2 69.48 40.43
68.75 71.8
81.9 76.3
70.5 76.8
68.59 75.25 63.2
68.6 76.7
71.1
73.7
? CO
100
CJ
oo
oo
o oo
o
CO
d
d d
S dd 1
10 d1 d
O Ö
,.-
ca
iela argi n, Co MaruI hun lém ent, B uthier ook
Jen est.
)6I
UOSIUl
"3 o
U
rj
SS
u (U U aQ QQ
403
•:;
p
a. —
»a
00 s '
w
ris
00
es
o
,08
T-^ CO — —'
183
367 648
vO OO
IT)
lO
.-'•o C > C/5 =3 1/5 X! OJ Q aiON u 5; •^ os «3 e s « b • '5ûda c a^
Q s tJN
KT
' • 3 X - " ; O
iU
^
¡ 2 2 '
8-e
N
rM iE
^a-
I I N-2
lll
00 «
e
>< s«
s M es .s oa -g S .S
ss
SUBJECTIVE AND OBJECTIVE MEMORY IN OLDER ADULTS c 0
1c
Êi
255
O
o o I
D
o I
d I
r)epress
c c
i ce •
331
Obi tive
:y
cti
u >
s o
• ^
u
/-
en ^ m' irf
(N
.- NO
^
^
— .
ive asu
s
scr een
e
~^
(S
en >; .2 .g °^
z
CS
ologica alidity
XI 3 Ù0
I -3 c o U-1 00 O^ fn
>^
o
a >Ä
ci 3
u m ON
—
C
© 2014 American Psychological Association 0882-7974/14/$l2.00 DOI: 10.!037/a0035908
Examining the Relationship Between Subjective and Objective Memory Performance in Older Adults: A Meta-Analysis Jessica J. Cramley, Cinnamon A. Stetler, and Michelle Horhota Furman University Are the beliefs that older adults hold about their memory abilities associated wi± their scores on lab-based memory tasks? A review of the aging literature suggests that the correlation between subjective and objective memory is inconsistent, with some studies reporting significant effects and others reporting null results. A meta-analysis was conducted to quantitatively summarize the relationship between subjective memory, defined as general predictions about memory, and objective memory performance in older adults, and to examine the conditions under which this relationship may be strongest. This meta-analysis included 53 studies, each of which included a normatively aging older adult sample. Overall, the association between subjective and objective memory was small (r = .062, SE = 0.014) but reliably greater than zero. Moderator analyses were conducted to better understand the parameters of this effect. Age, years of education, gender, depression symptoms, length and format of subjective memory measures, and type of objective memory were significantly correlated, with effect size. These results caution against relying on general subjective memory belief measures as a substitute for objective assessments of memory. Keywords: memory beliefs, memory complaint, memory performance, older adults, meta-analysis
tively impact an individual's well-being (Derouesné, Lacomblez, Thibault, & LePoncin, 1999; Mol et al., 2007; Verhaeghen, Geraerts, & Marcoen, 2000). Thus, subjective beliefs are an important component of the memory process that needs to be better understood. Although much empirical work has been done in this area, a consensus regarding the relationship between self-reported memory performance and objective memory performance in older adults has not emerged. Although some studies fail to demonstrate an association between subjective and objective memory (Cook & Marsiske, 2006; Derouesné et al, 1999; Mendes et al., 2008), other studies find significant correlations (Van Bergen, Jelicic, & Merckelbach, 2009). One reason for the inconsistent results may be that there actually is no relationship between general subjective beliefs and objective memory performance among older adults. Although memory does decline over time, the decline may be too subtle to self-assess accurately (Schmand, Jonker, Geerlings, & Lindeboom, 1997). Therefore, any attempt to characterize the relationship capitalizes on chance and random error (with some studies reporting true null findings). A second reason may be that the association is real but small. If studies are underpowered to find the effect, or if subjective memory is measured unreliably, it would lead to a high rate of Type II error. The present study uses meta-analytic techniques to determine whether the relationship between general subjective memory beliefs and objective memory is reliably nonzero, and if so, what the magnitude of the effect is. A meta-analysis is appropriate in this case because it provides a systematic review of the primary literature and provides a reliable quantitative synthesis of the available data that does not rely solely on the results of significance testing.
A large body of evidence suggests that Americans tend to hold negative views toward aging, with individuals expecting declines in competence (Kite, Stockdale, Whitely, & Johnson, 2005) and memory performance with age (Lineweaver & Hertzog, 1998). Older adults themselves typically report that although memory performance tends to decline over time, they personally believe that they will be equal to (or slightly better than) most other older adults (Lineweaver & Hertzog, 1998; Ryan & Kwong See, 1993). In actuality, individual differences and the amount of variability in older adults' cognitive abilities is large, (Zacks, Hasher, & Li, 2000) and everyone is most certainly not "better than average." This suggests a disconnect between how older adults perceive their memory performance and how they actually perform. It is important to understand whether older individuals are able to accurately assess their memory performance because individuals who are metacognitively aware are more likely to implement effective strategies, which can subsequently improve memory performance (Lachman & Andreoletti, 2006). Individuals who overor underestimate their memory abilities may not invest the required effort into a task, which can result in poor performance. These beliefs can also influence an individual's self-concept more broadly as they relate to anxiety about memory, which can nega-
Jessica J. Crumley, Cinnamon A. Stetler, and Michelle Horhota, Department of Psychology, Furman University. We thank Douglas Bonett for his advice and assistance with the metaanalytic methods and statistics. Correspondence concerning this article should be addressed to Michelle Horhota, Department of Psychology, Eurman University, 3300 Poinsett Hwy, Greenville, SC 29613. E-mail: [email protected] 250
SUBJECTIVE AND OBJECTIVE MEMORY IN OLDER ADULTS
Demographic Moderators Correlations between subjective memory and objective memory tend to be small and inconsistent. This may be due, in part, to demographic variables that may moderate the size of the effect. For example, age and education may be important to consider. Parisi et al. (2011) suggest that the older the sample, the larger the correlation between subjective and objective memory should be. Oldest-old adults may experience sizable cognitive declines that are readily perceived and are therefore more likely to be reported compared with the subtle objective declines that young-old adults may experience. Furthermore, the correlation may be larger when a study consists of a highly educated sample (Barker, Jones, & Jennison, 1995; Bolla, Lindgren, Bonaccorsy, & Bleeker, 1991; Zelinski, Bumight, & Lane, 2001). Those who are highly educated may have more insight into their own memory abilities (Zelinski et al, 2001), and also may be more likely to be aware of and use memory strategies to help compensate for any perceived decline (Lachman & Lachman, 1980). Thus, we predict that studies that include older or more educated samples will report larger associations between subjective and objective memory compared with studies with younger or less educated samples. Another potential moderator is the presence of depressive symptomology. Over 10% of adults over the age of 60 report suffering from an episode of depression in their lifetime, and many more experience elevated levels of negative affect or other symptoms that do not exceed a clinical threshold (Kessler et al., 2005). Studies conducted with normatively aging older adults have demonstrated that individuals with higher levels of depression tend to report higher levels of subjective memory complaints and lower levels of perceived memory ability (Bolla et al., 1991; Harwood, Barker, Ownby, Mullan, & Duara, 2004). Thus, it may be that elevated levels of depression, even in nonclinical samples, may minimize the relationship between subjective memory judgments and objective memory performance. Demographic variables such as gender and ethnicity have not been traditionally considered to moderate the relationship between subjective and objective memory. However, women and ethnic minorities are more likely to be influenced by variables that are known to moderate the relationship. For example, women and ethnic minorities tend to experience higher levels of depression (Kessler, 2006; Williams et al., 2007) and have lower education levels (Baker & Vêlez, 1996) than their Caucasian male counterparts. Because higher levels of education and lower levels of depression are thought to be associated with larger correlations, studies with larger proportions of women or ethnic minority members in their samples may yield smaller effect sizes than those studies with higher proportions of men or Caucasians.
Variations in Memory Measurements In the cognitive aging literature, subjective memory has been operationalized in a variety of different ways. For example, a distinction is made between "local" measures of subjective memory and "global" measures of subjective memory (Hertzog, Dixon, & Hultsch, 1990). Local measures are judgments that are specific to a context, and can be considered task-specific self-efficacy beliefs. For example, when participants make a prediction about their performance on a specific measure after receiving a brief description of the task, it would be considered a "local" or "task-
251
specific" subjective measure. Alternatively, global measures of subjective memory are more generalized and are removed from specific context (Hertzog et al, 1990). Global measures typically are assessed in the form of standardized questionnaires (e.g.. Memory Functioning Questionnaire [MFQ], Gilewski, Zehnski, & Schaie, 1990; Metamemory in Adulthood Questionnaire, Dixon, Hultsch, & Hertzog, 1988; Memory Assessment Clinics SelfRating Scale, Crook & Larrabee, 1992; Subjective Memory Complaints Questionnaire, Youn et a!., 2009; Cognitive Dysfunction Questionnaire, Vestergren, Ronnlund, Nyberg, & Nilsson, 2011). These measures typically aggregate responses across a range of memory types to create a general abstracted assessment of subjective memory ability. Although some global memory questionnaires include a "memory self-efficacy" factor, that factor tends to refer to general feelings of self-efficacy for memory tasks as a whole, rather than for specific memory tasks individually. Because global and task-specific measures assess different constructs, we focused our review on global performance judgment measures, and did not include studies that used only task-specific measures (see Beaudoin & Desrichards, 2011, for a review of this literature). Another important distinction in the literature is the timing of the subjective memory assessment. Subjective memory judgments can occur before the objective memory measure (a prediction) or after the objective memory measure (a postdiction). Past work has shown that the accuracy of a judgment is enhanced when subjective judgments follow performance of the memory task (Connor, Dunlosky, & Hertzog, 1997; Hertzog et al., 1990). Thus, postdictions refiect metacognitive monitoring accuracy, or how well a person monitors and updates their knowledge of task performance. In contrast, predictions reflect global beliefs in the absence of specific experience with the task (Connor et al, 1997). For the purpose of this meta-analysis, we focused solely on global predictions of performance, made prior to a participants' experience with a task, and excluded studies that focused on knowledge updating and monitoring. Even among global memory measures, important differences exist in the way the items are ñ'amed. Some global memory measures assess information about the participant's current perceived memory ability and functioning (e.g., Metamemory in Adulthood Questionnaire, Dixon et al., 1988; Multifactorial Memory Questionnaire, Troyer & Rich, 2002), whereas others tend to focus specifically on the frequency of memory failures and memory complaints (e.g.. Memory Assessment Clinics Self-Rating Scale; Crook & Larrabee, 1992). Research has suggested that the magnitude of the correlation between subjective and objective memory may vary depending on whether memory ability or memory complaints are used to assess subjective memory, with memory complaints correlating less strongly with memory performance (Balash et al., 2013; Hertzog & Pearman, in press; Zehnski, Gilewski, & Thompson, 1980). Global measures of subjective memory can also vary in their length and level of detail, as well as in format (questionnaire vs. interview). It may be the case that longer assessments more accurately reflect a person's subjective beliefs than shorter assessments, resulting in higher correlations with objective memory performance (Potter & Hartman, 2006; Reid & Maclullich, 2006; Wahlin, Maitland, Backman, & Dixon, 2003). Further, questionnaires tend to be more specific and longer than interviews (Jorm, Christensen, Körten, Jacomb, & Henderson, 2001). Interview questions are also less structured and more open-
CRUMLEY, STETLER, AND HORHOTA
252
ended (e.g., "Do you have any trouble remembering things that have happened recently?"; Jorm et al., 2001), and responses are coded by the interviewer on a scale of severity of complaint. Because questionnaires have more questions than interviews, and also tend to be more structured and less subject to interviewer biases, questionnaires may result in reliably larger correladons (Jungwirth et al., 2004). The way in which objective memory is assessed may also help to determine the magnitude of the effect size. Traditional lab-based memory tasks have been criticized for not being grounded in everyday memory performance (Sunderland, Harris, & Baddeley, 1983). Often, participants are assessed using objective measures with which they have no experience. Therefore, participants may rate their subjective memory based on everyday memory performance while being assessed on unfatniliar laboratory tasks that do not approximate daily life. Several studies suggest that the correlation between subjective and objective memory is higher when the study uses ecologically valid objective memory measures (e.g., shopping list recall or face name recall; Berry, West, & Dennehey, 1989; Schmidt, Berg, & Deelman, 2001; Sunderland, Watts, Baddeley, & Harris, 1986). Similarly, the type of memory assessed through objective measurement may influence the effect size. Individuals of all ages have different beliefs about the different forms of memory (Lineweaver & Hertzog, 1998). If personal memory beliefs vary across memory types, the correlations between beliefs and performance may differ based on memory type. Prospective memory, for example, seems to yield larger subjective and objective correlations compared with retrospective memory or working memory (Zeintl, Kliegel, Rast, & Zimprich, 2006). The reason for this may be that prospecfive memory performance is particularly salient to everyday life. Because of this, it is possible that prospective memory failures may be more easily recognized, and, therefore, older adults may be able to accurately track changes in prospective memory performance compared with episodic or working memory.
Current Study The goals of the current study are to first clarify the nature of any association between subjective and objective memory among normatively aging older adults by using meta-analytic techniques. Second, the study will determine whether sample characteristics, subjective memory measure variations, type of memory in the objective assessment, and level of ecological validity of objective memory measures influence the magnitude of the subjectiveobjective correlation.
Method Literature Search To obtain relevant studies, several search terms were entered into PsycINFO, MEDLINE, and Academic Search Premier databases. The first search field included the term "objective memory" or "memory performance." The second search field included the term "subjective memory" or "memory complaint" or "memory beliefs." The third search field included the term "older adults." In a fourth search field the following questionnaire names were included: "Memory Complaint Questionnaire" or "Memory Abil-
ity Questionnaire" or "Cognitive Failures Questionnaire" or "Metamemory in Adulthood Questionnaire" or "Memory Functioning Questionnaire." Our search covered published articles that were available prior to the end of February 2012. Reference secfions of review articles were consulted to identify relevant studies that were not identified through the computerized search. The authors also attempted to contact researchers in the field in order to obtain any unpublished findings. Studies were not restricted by date of publication or country of origin, but must have been published in English. Studies were included in the abstract screening process if they enrolled participants over 60 years of age. The search yielded 1,737 results whose abstracts were then reviewed for the following inclusion criteria.
Inclusion Criteria In order to be included in the meta-analysis, a study had to report at least one correlation between subjecfive memory, defined as a global prediction of memory performance in advance of completing a memory task, and objective memory. Fifty-seven percent of the studies were excluded because they did not measure both subjective and objective memory. Six percent of the studies were excluded because they did not report the correlation between subjective and objective measures of memory. Some studies included adults who were considered to have mild cognitive impairment (MCI) in their samples. The aim of this meta-analysis was to investigate the relationship between objective and subjective memory in normatively aging older adults; therefore, if these studies reported only one main effect, the proportion of participants within the sample with MCI had to be less than 30% in order to be included. Thiriy-three percent of the search results did not meet this criterion (i.e., included a sample that exceeded the 30% MCI limit). The cutoff of 30% was selected in order to balance the heterogeneity created by including clinical samples with our desire to include a generalizable sample of studies that was not overly narrow. If a study included over 30% MCI participants, but reported the correlation between subjective and objective memory for the non-MCI portion of their sample separately, only the effect size for the non-MCI sample was used. The final number of studies included in the current meta-analysis was 53. All studies are reported in Table 1 with their respective coding classifications. All coding was done by the first author. She consulted with the other authors in rare cases in which ambiguity made coding unclear, and coding was then based on consensus. Coding for IVIoderators Sample characteristics. The information that was extracted about each sample included age, gender composition (percent female), education level (in years), race/ethnic composition (percent Caucasian), mean depression score, and whether the study screened participants for MCI. The name of the depression measure that each study used was also recorded. In order to provide meaningful comparisons across different depression scales, depression means were standardized among the studies that used the same symptom measure. Because the presence of cogtiitive impairments could influence the association between objective and subjective memory, we coded each study on whether they screened and excluded people with MCI from analysis. A study was not
SUBJECTIVE AND OBJECTIVE MEMORY IN OLDER ADULTS included in this category if it was said to be testing normatively aging older adults but did not mention any type of screening procedure. This allowed us to determine whether samples known to be without cognitive impairment yield different effects compared with studies that only assume their samples are without cognitive impairment. Memory assessment. Subjective memory. The method used to assess global subjective memory was coded as being either an interview or questionnaire. To examine the effect of assessment length, assessments with 10 or fewer items were coded as "short." Assessments with more than 10 items were coded as "long." When possible, the subjective memory measures were categorized based on whether they assessed memory complaints, memory capacity, or both. A memory complaint directly yields a measure of subjective memory failures (e.g., "How often do names present a problem for you?"; Memory Functioning Questionnaire, Gilewski et al., 1990), whereas memory capacity captures one's perceived memory abilities (e.g., "I can remember the things I need to"; Memory Controllability Inventory, Lachman, Bandura, Weaver, & Elliott, 1995). Objective memory. Objective memory was coded based on the type of tnemory that was assessed: short-term, long-term, working, prospective, episodic (including paired-associates tasks), visual, semantic, and implicit memory as claimed by the author of a particular measure. Each measure was coded into only one category. The ecological validity of the objective memory measures was classified on the percentage of the objective tasks that approximated everyday memory performance. Tasks that were classified as everyday objective memory tasks included short story recall, face-name paired associates, facial recognition, shopping list recall, everyday object placement recall, noncomputerized prospective memory tasks (e.g., the blue card test, the red pencil task, call the lab tasks), photograph detail recall, celebrity name recall, lists of daily tasks recall (e.g., lists of appointments and errands), map/direction recall, doctor instructions recall, and personal recent and remote events recall. If a study's objective task battery contained one third or fewer everyday tasks, it was classified as a "low." A study was classified as a "moderate" if one third to two thirds of the objective memory tasks were ecologically valid, and as a "high" if more than two thirds of the objective tasks were ecologically valid.
Statistical Analysis The Pearson r correlation(s) between subjective and objective memory served as the dependent variable in all analyses (Rosenthal & DiMatteo, 2001). It is important for each smdy to contribute only one effect in order to meet the assumption of independent observations. If multiple correlations were reported within one study, the correlations were averaged to represent that study's overall effect. However, multiple effects within a study were retained when applicable and used in the relevant moderator analyses. Twenty-three of the 53 studies reported multiple effects, for a total of 87 unique effects. The overall mean effect (unweighted), standard error, and 95% confidence interval were computed under a varying coefficient (VC) model (Bonett, 2008). The VC model is an appropriate model because it does not assume homogeneity among effect sizes across studies (as a fixed effects
253
model does), nor does it assume the studies included in the meta-analysis are a random sample from some definable and normally distributed population of studies (as a random effects model does). Effects can be interpreted as significantly different from zero when the associated confidence interval does not include zero. All effect size, variance, and moderator calculations were completed with formulas from Bonett (2008) using SAS 9.2, except for the analysis of objective memory type, in whicb several studies reported multiple effects per study. In order to avoid the loss of data that would occur if those effects had been simply averaged, we employed hierarchical linear modeling techniques to account for the dependence in these data. Analyses involving this variable (objective memory type) were conducted using HLM 7.0 software.
Results Fifty-three studies representing 20,319 total participants were included in the meta-analysis. Information on each study can be found in Table 1. Publication dates ranged from 1975 to 2011 (mode = 2008). The average sample size was 383.38 {SD = 963.50, Mdn = 114, range = 20 to 6,772). Across all 53 smdies, correlations (Pearson r) between subjective and objective memory ranged from -0.290 to 0.410. Under a VC model (Bonett, 2008), the unweighted mean effect size was r = .062 {SE = 0.014, 95% CI [0.035, .090]). Although reliably greater than zero, the average correlation is very small in magnitude (Cohen, 1988). Subjective memory assessment explained less than 1% of the variance in objective memory performance. With an effect this small, many studies may not have found a significant effect and thus may have been less likely to be published. In order to determine whether the reported mean effect size could have been influenced by publication bias, we conducted three tests. First, we computed the number of unpublished studies with an effect size below 0.035 that would have to exist in order to reduce the average correlation below the level of significance. Using Orwin's (1983) fail-safe N formula, approximately 44 additional unpublished studies with null findings would have to exist. • Given that this number is approximately 83% the size of the current sample of studies, we believe that our reported mean effect size is unlikely to be only a product of publication bias. Second, we created a funnel plot (see Figure 1) to visually examine whether asymmetry due to publication bias was present. A visual inspection of the funnel plot does not reveal the typical pattern of asymmetry that would suggest the presence of publication bias. Twenty-nine of the effects are located to the left of the mean, and 24 of the effects are to the right of the mean. As a third measure of publication bias, we examined the association between estimates of effect size and their variances, as described by Begg and Mazumdar (1994); by this formal test, the current data set does not reveal publication bias, r = —0.032, p — .82. There is much debate in the literature surrounding the construction and interpretation of funnel plots (Tang & Liu, 2000; Terrin, Schmid, & Lau, 2005). The pattern of effects in Figure 1 is atypical in that the distribution does not resemble a funnel-like shape. Typically, a funnel shape emerges because larger, more precise studies, that is, those with small standard errors, cluster more tightly around the average effect size, whereas smaller, less precise studies yield more variable results. Instead, in our data set, studies
CRUMLEY, STETLER, AND HORHOTA
254 OO
CO 8 oo O
CO CO
IT)
(N
d d1
1
1
t'S r- es
d
Ö
d
d
o I
p
rs
(N es
S3
SS
Ö
II II
in SD
m' m'
o°
1^ Uu
O
O
SÖ
II es
^
o
CO
O
s
—
'S u
—.
(NI
(N —
_
_
_-
(-sj
-- —
(N
_
.-.
^
es
(D
d
81.63 63.64
64.7 80.23
72.84 63.2 69.48 40.43
68.75 71.8
81.9 76.3
70.5 76.8
68.59 75.25 63.2
68.6 76.7
71.1
73.7
? CO
100
CJ
oo
oo
o oo
o
CO
d
d d
S dd 1
10 d1 d
O Ö
,.-
ca
iela argi n, Co MaruI hun lém ent, B uthier ook
Jen est.
)6I
UOSIUl
"3 o
U
rj
SS
u (U U aQ QQ
403
•:;
p
a. —
»a
00 s '
w
ris
00
es
o
,08
T-^ CO — —'
183
367 648
vO OO
IT)
lO
.-'•o C > C/5 =3 1/5 X! OJ Q aiON u 5; •^ os «3 e s « b • '5ûda c a^
Q s tJN
KT
' • 3 X - " ; O
iU
^
¡ 2 2 '
8-e
N
rM iE
^a-
I I N-2
lll
00 «
e
>< s«
s M es .s oa -g S .S
ss
SUBJECTIVE AND OBJECTIVE MEMORY IN OLDER ADULTS c 0
1c
Êi
255
O
o o I
D
o I
d I
r)epress
c c
i ce •
331
Obi tive
:y
cti
u >
s o
• ^
u
/-
en ^ m' irf
(N
.- NO
^
^
— .
ive asu
s
scr een
e
~^
(S
en >; .2 .g °^
z
CS
ologica alidity
XI 3 Ù0
I -3 c o U-1 00 O^ fn
>^
o
a >Ä
ci 3
u m ON
—
C
