C International Psychogeriatric Association 2013 International Psychogeriatrics (2014), 26:3, 487–497  doi:10.1017/S1041610213002251

The development and validation of a patient-reported quality of life measure for people with mild cognitive impairment ...........................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................

Katherine Dean,1 Crispin Jenkinson,2 Gordon Wilcock3 and Zuzana Walker4 1

Nuffield Department of Medicine, University of Oxford, Oxford, UK Health Services Research Unit, Nuffield Department of Population Health, University of Oxford, Oxford, UK 3 Nuffield Department of Medicine, John Radcliffe Hospital, University of Oxford, Oxford, UK 4 Mental Health Sciences Unit, University College London, London, UK 2

ABSTRACT

Background: No validated patient-reported outcome measure (PROM) exists specifically to assess quality of life in mild cognitive impairment (MCI); we report a study conducted to develop such a measure. Methods: Semi-structured in-depth interviews were carried out with 23 people with MCI in order to determine items for a draft questionnaire. These interviews were audio-recorded, transcribed, and content analyzed. The draft questionnaire was refined following feedback from a focus group. 280 questionnaires were posted to subjects recruited from memory clinics and research databases, the response rate was 56% i.e. 146 questionnaires were included in the final analysis. The completed questionnaires were analyzed using factor analytic techniques to produce the final measure; construct validity was assessed by correlation with a generic patient-reported outcome measure, the SF-12v2. Results: Factor analysis produced a 13-item measure tapping two domains of patient-reported quality of life (“Emotional Effects” and “Practical Concerns”). Internal consistency reliability was high for both domains (α was 0.91 and 0.85 respectively). Both dimensions were highly and significantly correlated with the Mental Component Summary score of the SF-12v2 (“emotional effects” ρ = −0.43, p < 0.001 and “practical concerns” ρ = −0.56, p < 0.001). Conclusions: The Mild Cognitive Impairment Questionnaire (MCQ) is a 13-item measure developed specifically to measure patient-reported outcomes in people with MCI. It was created on the basis of patient report and has been shown to have good psychometric properties. It is likely to prove valuable in the evaluation of treatment regimes in this patient group. Key words: clinical assessment, cognitive impairment, mild cognitive impairment, quality of life (QoL)

Introduction Mild cognitive impairment (MCI) is a state that lies between normal cognition and dementia. The most commonly used diagnostic research criteria are those proposed by Petersen, i.e. that for a diagnosis there should be: memory complaint, normal activities of daily living (ADLs), normal general cognitive function, abnormal memory for age, and an absence of dementia (Petersen et al., 1999). There are a number of diagnostic criteria, in addition to Petersen’s original ones, in use in clinical practice: for example, the clinical criteria for the diagnosis of “MCI due to Alzheimer’s disease (AD)” developed by the National Institute Correspondence should be addressed to: Dr Katherine Dean, Nuffield Department of Medicine, University of Oxford, Level 5, John Radcliffe Hospital, Headley Way, Oxford, OX3 9DU. Email: [email protected]. Received 26 Jul 2013; revision requested 12 Aug 2013; revised version received 25 Oct 2013; accepted 3 Nov 2013. First published online 5 December 2013.

on Aging and Alzheimer’s Association working group (Albert et al., 2011) and those set out in the ICD-10 (World Health Organization, 2008). There has been some debate amongst clinicians as to the significance of MCI: some argue that it represents the “medicalisation” of age-related cognitive decline and that labeling such a condition as a medical disorder results in unnecessary distress for patients and their carers (Whitehouse, 2007). Others argue that true MCI simply represents a prodromal stage of dementia (Burgio, 1996; Palmer and Fratiglioni, 2006) and not a separate condition at all. It is estimated that there are about 1.5 million people in the UK with MCI (Smith et al., 2010). The conversion rates from MCI to dementia quoted in the literature are dependent on the definitions used and the populations studied (Ganguli et al., 2011) but the annual incidence of dementia seems to be around 3% in those with MCI in the general

488

K. Dean et al.

population (Ganguli et al., 2011) compared to approximately 0.5% in all people over 60 in the UK (National Institute for Clinical Excellence and Social Care Institute for Excellence (England and Wales), 2006). However, it is also known that a significant proportion of people meeting criteria for MCI show at least some improvement in cognition over time; studies have shown this to be in the case in 11–44% of those diagnosed with MCI at baseline (Wahlund et al., 2003; Gauthier et al., 2006; Matthews et al., 2008). Some people diagnosed with MCI may return to entirely normal cognition over time. Until recently, the majority of interest in MCI has been due to the increased risk of dementia that the diagnosis carries; however, there is now increasing interest in MCI as an entity in itself and quite a number of recent studies have explored the experiences of those living with MCI. These studies have provided evidence that those living with MCI face a range of cognitive, emotional, and practical challenges of a greater magnitude than might be expected given the diagnostic criteria described above. Understandably, cognitive complaints are common in MCI: one study involving guided interviews of eight very recently diagnosed patients in Holland identified four common themes (Joosten-Weyn Banningh et al., 2008) one of which was “changes noticed by the patient” – these were often cognitive in nature, for example “forgetfulness” and “poor concentration.” Frank et al. carried out focus group interviews of 20 people with MCI and 11 “informants,” as well as similar numbers of people with mild probable AD (Alzheimer’s disease) (Frank et al., 2006b). They also found that PWMCI commonly reported cognitive changes affecting recall, verbal fluency, and processing skills. In a third qualitative study, Lu et al. carried out open-ended interviews with 11 PWMCI in order to identify “commonalities of the lived experience of being diagnosed and living with MCI” (Lu et al., 2007). They described a longitudinal “journey” experienced by PWMCI beginning with the gradual awareness of their problems, followed by efforts to maintain a sense of “being able,” and finally development of strategies to maintain “a sense of self.” The first of these elements incorporated an awareness of worsening memory. The issue of subjective memory complaint (SMC) and its relationship to MCI has attracted some attention in the literature. SMC is common and its significance is often unclear; a meta-analysis of studies of the clinical significance of SMC found that only approximately 30% of patients with SMC included in the studies went on to receive a diagnosis of MCI (Mitchell, 2008). Conversely, it may also

be the case that some people with MCI underreport their difficulties due to impaired insight. There is conflicting evidence regarding this topic but one study in which individuals with MCI were compared to those with mild AD and controls found that the subjects with MCI and mild AD had a similar degree of impaired insight into their memory problems and that this was greater than in controls (Vogel et al., 2004). Another commonly reported problem in MCI is functional impairment: Despite the stipulation in the original Petersen definition of MCI that basic ADLs should be intact it has been increasingly recognized that more complex (or “instrumental”) ADLs (IADLs), such as managing medications or finances, may be affected by MCI. Difficulties with IADLs in MCI have been identified in quite a number of studies, often at a rate between those experienced by people with dementia and normal controls: In a large study based on data from the Alzheimer’s Disease Neuroimaging Initiative (ADNI) informant reports of function (as measured with the quantitative Pfeffer Functional Activities Questionnaire) for nearly 400 PWMCI were compared with those for controls and subjects with mild AD (Brown et al., 2011). The authors found that functional impairment in one or more of the domains covered by the questionnaire was present in 72% of the PWMCI compared to 8% controls and 95% subjects with mild AD and that, perhaps unsurprisingly, greater functional impairment was associated with poorer performance on various neuropsychological tests. Neuropsychiatric symptoms (NPS) are also known to be a common feature of MCI: In a population-based cohort study of PWMCI, carried out as part of the Cardiovascular Health Study in the United States, Neuropsychiatric Inventory (NPI) scores were examined for 320 people classified as having MCI. Forty-three percent were found to have had at least one NPS in the preceding month with the commonest symptoms being of depression (20%), apathy (15%), and irritability (15%) (Lyketsos et al., 2002). Results of some of the qualitative studies in this area also appear to confirm that negative emotional reactions are common in PWMCI. For example, one of the major themes identified in Joosten-Weyn’s interview-based study was “consequences” which were commonly negative emotions experienced by PWMCI regarding their cognitive problems (Joosten-Weyn Banningh et al., 2008). The subjects in Lu et al.’s study also described a range of negative emotional reactions to their memory problems (Lu et al., 2007). There are a myriad of tools, both generic and disease specific which have been developed to assess health-related quality of life in various patient

A patient-reported outcome measure for MCI

groups; detailed discussion of these is beyond the scope of this paper. The SF-36 and the short-form version, the SF-12, are among the most widely used of the generic measures, they are multi-purpose health surveys which provide measures of functional health and well-being as well as psychometrically based summary measures of physical and mental health (the Physical Component Summary (PCS) and Mental Component Summary (MCS) respectively). Despite the plethora of tools which have been developed for generic use and use in other diseases, there is currently no consensus on the best way to assess needs or outcomes in MCI. A recent review of needs assessment instruments for use in mental and cognitive disorders concluded that “there is a lack of instruments to assess the needs in individuals with subjective cognitive impairment (SCI), MCI or dementia comprehensively” (Schmid et al., 2012). The authors identified and reviewed 17 instruments, most of which took the form of structured interviews or questionnaires; the majority were designed to assess the needs of patients with dementia or long-term mental illness and none were designed specifically for use in MCI. The authors noted that, as patient-reported unmet needs (such as those experienced by people with MCI described above) have been shown to correlate negatively with quality of life outcomes (QoL), tools designed to assess specific needs and hence guide appropriate interventions should be developed. A review of patient self-reported measures for use specifically in MCI identified a small number of patient-reported outcome measures (PROMs) used for the assessment of everyday function/ADLs, executive functioning, neuropsychiatric symptoms, and health-related quality of life (HRQL) (Frank et al., 2011). With the exception of the “Patient Reported Outcomes in Cognitive Impairment” (PROCOG) tool (which is discussed below), all other instruments identified were developed for use in other conditions (usually dementia) and adapted/validated to varying degrees for use in MCI. The issues affecting quality of life in MCI and dementia are likely to be different in a number of ways: By definition, people with MCI have less pronounced cognitive and functional impairment than those with dementia and, as discussed above, they may retain a greater degree of insight into their limitations. There is a greater degree of uncertainty associated with a diagnosis of MCI given the heterogeneous range of outcomes for this group and the fact that the general public are much less familiar with the concept of MCI than they are with dementia. People with MCI are also much less likely to exhibit any of the severe behavioral disturbances which can be present in dementia and have been shown to be an important determinant of quality of

489

life in this group (Burgio, 1996; Van der Mussele et al., 2013). It is by no means certain, therefore, that measures developed to measure HRQL in dementia will be valid for use in MCI. There has been recent increasing interest in the use of patient-reported outcome measures (PROMs) to provide information on the impact of disease in clinical practice and to measure outcomes both in clinical practice and clinical trials. PROMs are defined by the UK Department of Health as “measures of a patient’s health status or healthrelated quality of life . . . typically short, self-completed questionnaires.” In 2009, the United States Food and Drug Agency (FDA) published guidance on the use of PROMs which set out the “best practice” points which they consider when reviewing such instruments and guidance on the development of new measures (US Department of Health and Human Services et al., 2009). Only one PROM, the PROCOG (Frank et al., 2006a), has been developed and validated specifically for use in MCI. This instrument, however, does not measure QoL specifically, rather it is a “symptom impact questionnaire” which the developers suggest might be used as part of the neuropsychological diagnostic process and in the assessment of needs for PWMCI. In addition, the sample studied during development of the PROCOG comprised both people with MCI and AD rather than MCI alone. The evidence reviewed above suggests that PWMCI face significant emotional and practical burdens which are likely to impact adversely on QoL in this group. Regardless of the controversy surrounding the merits of the diagnostic label of MCI it remains a fact that MCI is being diagnosed with increasing frequency in the UK and therefore there are increasing numbers of people living with this diagnosis. In light of this and the lack of existing measures for use in this group use we felt it was important that a measure of health-related quality of life be developed for use specifically in MCI. Therefore, the aim of this study was to develop a patient-reported outcome measure to assess healthrelated quality of life in people with MCI (the Mild Cognitive Impairment Questionnaire or MCQ), in line with FDA guidance where possible; this paper reports its development and validation.

Methods Stage 1 – Item generation In depth, semi-structured interviews with 23 people recently diagnosed with MCI were carried out. Topic guides for the interviews were derived from a review of the existing literature and discussion with an expert panel comprised of a professor

490

K. Dean et al.

of Geriatric Medicine with a specialist interest in memory disorders, a consultant old age psychiatrist, and a professor of Health Services Research with a specialist interest in the measurement of healthrelated quality of life (HRQL) and the evaluation of patient experiences of medical care. The final topic guides covered aspects of the day-to-day experience of living with MCI. The inclusion criteria for this stage of the study were: being aged 50 years or older, having been diagnosed with MCI (using whichever criteria the diagnosing clinician applied) within a secondary care memory service within the preceding six months, and being able to converse fluently in English. A purposive sample of PWMCI was identified from memory clinics (and “analogous services”) in the UK in Oxfordshire, Buckinghamshire, Berkshire, Northamptonshire, and Essex, from research databases in Oxfordshire and Essex and via advertisements (in the form of posters and patient leaflets) placed in clinical areas in these regions. “Analogous services” were defined as “equivalent memory services being provided outside the ‘traditional’ setting of a hospital outpatient memory clinic,” for example, services provided in the patient’s home by community mental health teams (CMHTs). Patients recruited from clinic were approached at their appointment with initial information about the study. If they expressed an interest in taking part in the study consent was obtained to screen their clinical notes to confirm eligibility for the study. Eligible patients were sent full study information and contacted by telephone to arrange an appointment for interview. Patients identified from the database were sent full information about the study by post and asked to return a form indicating consent to screening of their clinical notes if they were interested in taking part in the study; recruitment then proceeded as for patients recruited from clinic. PWMCI who contacted the research team having seen an advertisement for the study were sent full study information by post and recruitment proceeded as for PWMCI identified from the research databases as described above. All subjects in this part of the study gave written, informed consent for participation. Ethical approval for both parts of the study was obtained from a UK research ethics committee and the Research and Development departments of each hospital trust involved. The sample size for this part of the study was determined by the point at which data saturation was reached, i.e. the point at which new interview data provided no new information in terms of themes identified by qualitative data analysis.

Participants in this stage of the study were aged between 63 and 86 years old with a mean age of 77.8 years (SD 6.2 years). Thirteen interviewees were male and 10 were female, all were of White British origin. Mini-Mental State Examination (MMSE) (Folstein et al., 1975) scores were available for 14 of the subjects interviewed, the mean MMSE for these subjects was 26.9 (range 22–30, SD 3.0). The interviews were audio-recorded and transcribed verbatim; data were entered into NViVo software and the most frequently recurring and salient themes were identified. These were as follows: Effects of challenges presented by MCI: Impaired recall (for recent events, conversations, names of friends, future plans such as appointments, and location of items around the house), verbal difficulties (object naming and impaired verbal fluency – both spoken and written). Impact on daily life: Difficulty managing personal paperwork and finances and detrimental effect on social life/hobbies. Effect on relationships: detrimental impact on relationship with spouse/family, increasing reliance on spouse, feeling unable to discuss their condition with spouse/family/friends, and comparison with others’ memory. Perceived change in personality: decreased self-confidence and feeling “generally slowed down.” Negative emotional reactions: Irritation, frustration, anxiety (about current condition/future), sadness, and embarrassment/concern about others’ reactions. A set of preliminary questionnaire items was drafted based on the themes, these were discussed with the expert panel (as described above) and refined to produce a draft questionnaire. The draft questionnaire was discussed with a focus group of 11 people with MCI who were identified from a local research database and sent an invitation to attend an informal discussion about a new questionnaire being developed for use in MCI. The PWMCI attending the focus group had a mean age of 75.9 years (range 63–86, SD 7.4 years), 7 were male and 4 were female. The focus group participants were shown the draft questionnaire and asked for comments about its clarity and readability and whether all salient points appeared to have been covered. The focus group’s comments were audio-recorded and analyzed, amendments were made to the questionnaire based on the feedback. The resulting final draft version of the questionnaire was laid out as a set of 17 statements describing phenomena that the respondent might have experienced with a Likert type rating scale for the subject to complete (Likert, 1932). The response options given were: “never/rarely/sometimes/often/always.” The statements were written in language reflecting that used by interview subjects wherever possible.

A patient-reported outcome measure for MCI

A previously validated generic measure of health status, the Medical Outcomes Study 12-Item ShortForm Health Survey (SF-12v2) (Ware et al., 2007), and a set of questions about the subject’s demographic details were also included in the survey pack. Stage 2 – Item reduction, scale generation, and testing of construct validity The questionnaire was posted to people with MCI listed on 11 research databases in the south of England and given out in memory clinics in the seven participating healthcare trusts, also in the south of England. Inclusion criteria for subjects in this part of the study were: being 50 years of age or older, having had a diagnosis of MCI confirmed (using whichever criteria the diagnosing clinician applied) at a secondary care memory service within the past 12 months, and being able to read and write in English. “Recruitment packs” were given out at memory clinics (and “analogous services”) in Oxfordshire, Buckinghamshire, Northamptonshire, Leicestershire, and North London to eligible patients who had been given brief verbal information about the study and had indicated that they were interested in taking part. The recruitment packs included a patient information sheet, the patient questionnaire, and a pre-paid envelope in which to return the questionnaire. The patient was asked to complete the questionnaire (either in clinic or at home) and to return it in the pre-paid envelope. “Recruitment packs” were also sent out to eligible subjects on research databases in Oxfordshire, Essex, Buckinghamshire, Berkshire, Northamptonshire, Avon and Wiltshire, and London. These recruitment packs included the same material as those used in clinic together with a covering letter explaining why the pack had been sent. Those subjects who had not replied within two weeks were sent a reminder letter about the study. The aim was to obtain at least 100 completed questionnaires to ensure a sufficient sample size to allow evaluation of the psychometric properties of the MCQ. This follows recommendations for acceptable sample sizes when developing this type of instrument which suggest that 3–5 responses per questionnaire item are obtained, with a minimum of 100 in total (Kline, 1999; Costello and Osborne, 2005). Two hundred and eighty questionnaires were supplied to potential participants, 150 completed questionnaires (54%) were returned. Four subjects were excluded as they did not meet the inclusion criteria and therefore 146 eligible subjects were included in the analysis. The subjects ranged in age from 52 to 91 years with a mean of 75.4 years

491

and a standard deviation (SD) of 7.6 years. 57.3% subjects were male and 42.7% were female. Statistical analysis Descriptive analyses of demographic data and SF12v2 Mental Health Component Summary Scores (MCS) and Physical Component Summary Scores (PCS) were performed using SPSS 18.0. Measure development and validation Items with high floor or ceiling effects were removed, a guideline of greater than 40% respondents selecting “always” or “never” was used for this. Items with greater than 10% missing data were also removed. Correlations were calculated between items to ensure none were very highly correlated with each other and therefore tapping the same issue. The data were then analyzed using exploratory factor analysis in which items most strongly interrelated tend to gain high loadings on a single factor suggesting they are assessing the same underlying concepts. Varimax rotation was used to produce an orthogonal solution. Factors with an eigenvalue of greater than 1 were retained in the analysis; items with a loading of less than 0.40 on a factor were removed from the factor. Cronbach’s α statistic (Cronbach, 1951) was used to assess the internal consistency of each domain identified by the factor analysis. The total item scores for each domain identified by the factor analysis were transformed to create a scale score as follows: scale score = (total of the raw scores of each item in the scale minus the number of items in the scale) divided by (the maximum possible raw score of all the items in the scale minus the number of items in the scale) multiplied by 100. Scale scores ranged from 0 (best, i.e. no problem at all) to 100 (worst, i.e. maximum level of problem). The construct validity of the scales of the MCQ was examined by correlation with the SF-12v2 MCS and PCS. The SF-12v2 has 12 questions with Likert-type response options, e.g. “During the past 4 weeks, how much of the time has your physical health or emotional problems interfered with your social activities (like visiting friends, relatives etc.) – all/most/some/a little/none of the time?.” The responses yield an eightscale profile of functional health and well-being. Factor analyses of correlations among the eight health domain scales have consistently identified two factors – the basis of the PCS and MCS summary scores which are also produced by the SF12v2 (the PCS and MCS respectively) (Ware et al., 2007). The SF-12 was chosen as an instrument against which to measure construct validity as it has been well validated, including for postal use and extensively used in the past (Ware et al., 1996; Lee et al., 2008; Martinez et al., 2008; Chen et al.,

492

K. Dean et al.

Table 1. Rotated component matrix resulting from factor analysis of questionnaire data ITEM

COMPONENT

1

COMPONENT

2

............................................................................................................................................................................................................................................................................................................................

Worry that you have forgotten things such as recent conversations or the names of things or people Worry that you have had problems constructing a sentence when talking Worry that you have forgotten what you had planned to do Worry that you have had problems remembering appointments or important dates, such as birthdays Worry about feeling generally “slowed down” Worry that you have upset other people because of your memory problems Feeling that you have become less independent because you have had to rely on your partner or other people to help you remember things Irritation or frustration about your memory problems Feeling worried about your memory problems Feeling downhearted or depressed about your memory problems Worry about other people’s reactions to your memory problems Worry that your memory problems are more severe than those of other people of your age Worry about your memory problems getting worse in the future

0.382

0.661

0.261 0.335 0.174

0.577 0.714 0.794

0.308 0.362 0.407

0.632 0.623 0.650

0.723 0.823 0.826 0.606 0.716

0.356 0.351 0.336 0.403 0.367

0.825

0.250

Note: Extraction method – principal component analysis; rotation method – Varimax with Kaiser normalization; rotation converged in three iterations.

2009). The MCS and PCS have been shown to have good internal consistency reliability with reliability coefficients of 0.87 and 0.91 for the MCS and PCS respectively (Ware et al., 2007). As the majority of the items of the MCQ pertain to the emotional impact of the condition it was hypothesized that scores on the MCQ would be more highly correlated with the MCS and have low correlations with the PCS.

Results Item reduction and scale generation No items had >10% missing data and none were very highly correlated with other items (using a cut-off as 0.8 to define “very highly correlated”) suggesting that no items were tapping the same issues. The item intercorrelations are given in Table S1 published as supplementary material online, attached to the electronic version of this paper at http://journals.cambridge.org/ipg. Four items were removed because of floor effects, these were (% respondents selecting “never” given in brackets): ‘not being able to take part in hobbies or social events’ (42.8%), ‘feeling that you have had to cover up your memory problems to avoid upsetting someone’ (51.1%), ‘worry about a change in your personality’ (39.3%), and ‘feeling unable to talk to friends or relatives about your memory problems because it is upsetting or embarrassing’ (38.6%). A factor analysis was performed on the remaining 13 items; two domains were identified accounting for 61.8% of the total variance. The results of the factor analysis are given in Table 1.

The two domains identified were as follows: Emotional effects (6 items): This domain addresses the emotional effects of living with MCI including irritation/frustration, anxiety, low mood, concern about the future, worry about the reactions of others, and worry that their memory problems are more severe than those of their peers. Practical concerns (7 items): This domain covers the practical effects of living with MCI including worry about: having forgotten things (e.g. names), plans, or appointments, problems with conversation due to memory difficulties, feeling generally “slowed down” or less independent, and concern about upsetting others. The two factor solution was also supported by the scree plot produced by factor analysis, this is given in Figure S1 published as supplementary material online, attached to the electronic version of this paper at http://journals.cambridge.org/ipg. Both domains were transformed to give scales with a range from 0 (indicating no problem) to 100 (indicating maximum level of problem) as described in the “Methods” section. The questionnaire data were used to calculate the mean, standard deviation and range of scale scores for the domains of the MCQ and the SF-12v2 MCS for the study sample as a whole; MCQ domain scores were also calculated separately for male and female subjects and for those subjects aged less than 77 years (the median age of the study sample) and aged 77 years or older. These values are shown in Table 2. The most pronounced difference between subgroups was between male and female subjects’

A patient-reported outcome measure for MCI

493

Table 2. Descriptive statistics for the MCQ scales and SF-12v2 MCS

SCALE

SUBGROUP

MEAN (SD)

N

N (%) SCORING MINIMUM SCORE

RANGE OF SCORES

N (%) SCORING MAXIMUM SCORE

D I FF E R E N C E B E T W E E N SUBGROUPS: MEAN (SE) AND ∗ S I G N I FI C A N C E

............................................................................................................................................................................................................................................................................................................................

Emotional effects

Practical concerns

All subjects

47.6 (24.2)

137

0–100

2 (1.5)

1 (0.7)

Male subjects

47.4 (25.3)

80

0–100

1 (1.2)

1 (1.2)

Female subjects Subjects 0.05

0.05

∗ Calculated using the Mann-Whitney U test. Note: SD = standard deviation; N = number; SE = standard error.

Table 3. Correlation of MCQ items to total correlations and internal reliability of domains (Cronbach’s α) DOMAIN

CORRECTED ITEM TO TOTAL CORRELATION

ITEMS

CRONBACH’S α

............................................................................................................................................................................................................................................................................................................................

Emotional effects

Practical concerns

Irritation or frustration about memory

0.71

0.91

Worry about memory Feeling downhearted about memory Worry about others’ reactions to memory problems Worry that memory is worse than peers Worry about memory worsening in the future Worry about forgetting things, e.g. names

0.82 0.82 0.64 0.73 0.77 0.64

0.85

Worry about sentence construction Worry about forgetting plans Worry about forgetting appointments Worry about feeling slowed down Worry about upsetting others because of memory problems Feeling less independent

0.51 0.67 0.65 0.59 0.61 0.66

“practical concerns” scale scores, with male subjects tending to have higher scores (indicating a greater degree of problem in this domain). However, none of the differences between scale scores for the subgroups were statistically significant. The correlations of items to their scale totals and the internal consistency reliability of the domains are shown in Table 3. Both the scales have high internal

consistency reliability by the accepted standard (Cronbach, 1951; Nunnally and Bernstein, 1994; Streiner and Norman, 2003). Evaluating construct validity The construct validity of the scales of the MCQ was examined by correlation with the SF-12v2 MCS and PCS. The “emotional effects” scale was

494

K. Dean et al.

correlated with the MCS (ρ = −0.43, p < 0.001, N = 111) and did not correlate significantly with the PCS (ρ = −0.11, p > 0.02, N = 111). The “practical concerns” scale was correlated with the MCS (ρ = −0.56, p < 0.001, N = 113) more highly than with the PCS (ρ = −0.34, p < 0.001, N = 113). The final version of the MCQ is given in Figure S2 published as supplementary material online, attached to the electronic version of this paper at http://journals.cambridge.org/ipg.

Discussion Evidence from quantitative and qualitative studies, such as those described in the introduction, suggests that PWMCI experience a range of difficulties and to a greater extent than might be expected given the general consensus that MCI should have “an absence of major repercussions on everyday life” (Portet et al., 2006). Despite this, the MCQ is the first measure designed specifically to assess patientreported QoL in PWMCI. As mentioned in the introduction, one other instrument, the PROCOG (Frank et al., 2006a) has been developed for use in people with cognitive impairment but this includes both PWMCI and with mild AD. Furthermore, the PROCOG is primarily a symptom impact measure. The results discussed above indicate that the MCQ has good psychometric properties: The Cronbach’s α statistics for both domains indicate good internal consistency reliability. The correlation between scores for the SF-12v2 MCS and the MCQ scales and the fact that the MCQ scales correlated more strongly with the MCS than PCS confirm construct validity. The negative correlations reflect the fact that the MCS/PCS and the MCQ scale scores operate in different directions: a higher MCS/PCS indicates “better” health whereas higher MCQ scale scores indicate a higher degree of difficulty in that domain. Although there are no published studies in which it has been validated for use specifically in people with cognitive impairment, the SF-12 has been shown to be psychometrically sound for assessing QoL in people with severe mental illness such as psychosis and major mood disorders (Salyers et al., 2000). In addition, it has been shown that the SF-36 and its component subscales have reasonable psychometric properties for assessing QoL in people with mild to moderate dementia (specifically those with a Mini-Mental State Examination score of 15 or greater) (Novella et al., 2001) and that there is a high degree of correspondence between summary physical and mental health measures (i.e. the PCS and MCS) estimated using the SF-12 and SF-36 (Gandek et al., 1998).

Hence, in the absence of any existing measures of quality of life validated specifically for use in MCI, the SF-12 and its component summary measures are reasonable tools against which to assess construct validity. Content validity is demonstrated by the fact that the measures were developed based on the results of in-depth interviews with PWMCI. The content of the measure reflects the varied issues which are relevant to PWMCI: many of the items relate to the effects of cognitive changes noticed by the subject, neuropsychiatric symptoms, and the impact of functional limitations – these categories reflect those commonly discussed in the literature about the experiences of those living with MCI cited in the introduction to this paper. In addition, the MCQ includes items relating to the subject’s interaction with others (specifically items 6, 11, and 12). The two domains identified by factor analysis (emotional effects and practical concerns) confirm that the issues which affect QoL in MCI can broadly be divided into the effects of emotional/neuropsychiatric symptoms and the impact of limitations in function. Many of the items included in the MCQ reflect those in the PROCOG but there are some interesting differences: Item 1 of the MCQ (“worry that you have forgotten things such as recent conversations or the names of things or people”) describes a concept which the PROCOG uses 9 items to assess (items 23–28 and 30–32). Many of the other MCQ items appear to equate to one or two items on the PROCOG; however, several MCQ items appear to have no equivalent in the PROCOG and thus apparently measure concepts not covered by the PROCOG. MCQ items with no equivalent in the PROCOG are: item 4 (“worry that you have had problems remembering appointments or important dates, such as birthdays”), item 5 (‘worry about feeling generally ‘slowed down”), and item 12 (“worry that your memory problems are more severe than those of other people of your age”). Two MCQ items seem to have similar, although not necessarily equivalent, items in the PROCOG: Item 6 (“worry that you have upset other people because of your memory problems”) is similar to PROCOG item 49 (“do you think friends call you less than they used to because of your memory or thinking problems?”) although the PROCOG item does not incorporate the concept of being concerned about “upsetting” other people. Also, item 7 (“feeling you have become less independent because you have had to rely on your partner or other people to help you remember things”) is similar to PROCOG item 50 (“because of your memory or thinking problems, have family routines changed, like who does most of the driving or cooking or who usually handles finances”) – although the PROCOG item relates to

A patient-reported outcome measure for MCI

practical impact whereas the MCQ item focuses on the effect that practical changes have had on the subject. Overall, in comparison to the PROCOG, the MCQ is a more concise instrument; it also appears to assess some additional concepts and has a greater focus on the emotional impact of changes noticed by the subject, rather than simply quantifying the changes themselves. The methods used to develop the questionnaire were chosen as they have been shown to be effective in the development of similar questionnaires in the past (Jones et al., 2001; Jenkinson et al., 2012) and, in many areas, were compliant with the FDA guidance on the development of PROMs mentioned in the introduction (US Department of Health and Human Services et al., 2009). PWMCI were recruited from memory clinics (and analogous services) and research databases in various regions which ensured that the study population was geographically diverse and had had contact with a range of different healthcare services. There are some potential limitations to the study: The inclusion of subjects listed on research databases allowed recruitment of a sufficiently large sample for the purposes of the study, it may, however, have introduced a degree of selection bias, as some of these subjects had taken part in other research projects which, whilst not part of their standard clinical care, may have influenced their experiences of living with MCI and hence their questionnaire answers. Another possible limitation of the study is the use of the inclusion criterion that subjects should have a diagnosis of MCI confirmed “using whichever criteria the diagnosing clinician applied”: this is likely to have resulted in a study sample which is heterogeneous in terms of MCI type which might make the results difficult to apply to research populations where a particular MCI definition is applied. However, this approach does have the advantage of producing results which are applicable to general “real world” clinic populations, where MCI groups are likely to be heterogeneous, and therefore maximizes the potential for practical application of the MCQ in this setting. As discussed in the introduction, it may be that some PWMCI have impaired insight into their condition and are therefore likely to underreport their difficulties. As results of studies in this area have been conflicting, it is difficult to know how this phenomenon might affect the validity of the MCQ – although it is hoped that the removal of items with high floor effects has left a set of items relating to issues about which most PWMCI will retain insight. In a study of awareness of deficits in MCI, Roberts and Clare used a qualitative approach (using interpretative phenomenological analysis) to gain a more in-depth understanding of “meta-

495

representational” awareness of symptoms and their impact in PWMCI (Roberts and Clare, 2013). They defined “meta-representational awareness” as the most complex expression of awareness – which “encompasses aspects of self-identity and the environment.” They concluded that all the PWMCI they interviewed did display awareness of their cognitive difficulties at this level but that various psychological and social factors influenced how this awareness was expressed. It may be that a PROM such as the MCQ will enable PWMCI to express the awareness of their deficits which Roberts and Clare’s study suggests they retain. The MCQ is a short, simple assessment tool which could be used easily in variety of clinical and research settings to assess the effect of interventions for PWMCI. At present, the assessment of interventions in MCI is hampered by the lack of consensus regarding which outcome measures to use in interventional trials making the results difficult to interpret and compare (Frank et al., 2011). The MCQ is a potential solution to this: for example, it could have a role in elucidating the effectiveness of single and multi-component support programs, trials of which, up to now, have produced conflicting and difficult to interpret results (Lautenschlager et al., 2010; Joosten-Weyn Banningh et al., 2011), due in part to the variation in outcome measures used. In light of the FDA guidance on the use of PROMs in drug trials, it is also possible that the MCQ could play a role in the assessment of pharmacological interventions developed for use in MCI in the future. Use of the MCQ for these purposes has the potential to improve the assessment of interventions for MCI and therefore help clinicians decide which of the potential interventions, if any, result in meaningful improvements in outcomes in this group. Further work to establish the responsiveness of the MCQ (in line with the FDA guidance on the topic) is planned in order to facilitate its use as an outcome measure in interventional trials. Another potential application is in the assessment of progress of individual patients in clinical practice – this too would require further assessment of the measure. Use of the MCQ for this purpose would allow clinicians to track patients’ progress terms of the issues which are known to have a significant impact on their QoL; this may help them to decide on the type of intervention or support a particular patient might require. In summary, the MCQ is, to our knowledge, the first PROM developed specifically to assess quality of life in people with MCI; it has been shown in this study to have good psychometric properties. The MCQ has several potential future applications which, it is hoped, will contribute to

496

K. Dean et al.

improvements in the help and support provided to people diagnosed with MCI. Further information on licensing and use of the questionnaire is available from Isis Outcomes (http://www.isis-innovation.com/outcomes/), those interested in using the MCQ should contact Isis Outcomes.

Conflict of interest The study was funded by Bupa Giving. The MCQ is under copyright by Isis Innovation Ltd. KD, CJ, and ZW may gain financially should it be used for commercial purposes.

Description of authors’ roles K. Dean assisted in designing the study, carried out the study, assisted in analyzing the data, and wrote the paper; C. Jenkinson assisted in designing the study, assisted in analyzing the data, and assisted in writing the paper; G. Wilcock assisted in designing the study and writing the paper; and Z. Walker assisted in designing and carrying out the study and in writing the paper.

Acknowledgments The authors thank all the research staff who helped with recruitment to this study including those within Avon and Wiltshire Mental Health Partnership NHS Foundation Trust, Barnet Enfield and Haringey Mental Health NHS Trust, Berkshire Healthcare NHS Foundation Trust, Camden and Islington NHS Foundation Trust, Imperial College Healthcare NHS Foundation Trust, Leicestershire Partnership NHS Trust, North Essex Partnership NHS Foundation Trust, Northamptonshire Healthcare NHS Foundation Trust, Oxford Health NHS Foundation Trust, Oxford University Hospitals NHS Trust, and West London Mental Health NHS Trust.

References Albert, M. S. et al. (2011). The diagnosis of mild cognitive impairment due to Alzheimer’s disease: recommendations from the National Institute on Aging-Alzheimer’s Association workgroups on diagnostic guidelines for Alzheimer’s disease. Alzheimers Dement, 7, 270–279. Brown, P. J., Devanand, D. P., Liu, X. and Caccappolo, E. (2011). Functional impairment in elderly patients with mild cognitive impairment and mild Alzheimer disease. Archives of General Psychiatry, 68, 617–626. Burgio, L. (1996). Interventions for the behavioral complications of Alzheimer’s disease: behavioral approaches. International Psychogeriatrics, 8 (Suppl. 1), 45–52.

Chen, P. H., Golub, J. S., Hapner, E. R. and Johns, M. M., 3rd. (2009). Prevalence of perceived dysphagia and quality-of-life impairment in a geriatric population. Dysphagia, 24, 1–6. Costello, A. B. and Osborne, J. W. (2005). Best practices in exploratory factor analysis: four recommendations for getting the most from your analysis. Practical Assessment, Research and Evaluation, 10, 173–178. Cronbach, L. (1951). Coefficient alpha and the internal structure of tests. Psychometrica, 16, 297–334. Folstein, M. F., Folstein, S. E. and McHugh, P. R. (1975). “Mini-mental state”: a practical method for grading the cognitive state of patients for the clinician. Journal of Psychiatric Research, 12, 189–198. Frank, L., Lenderking, W. R., Howard, K. and Cantillon, M. (2011). Patient self-report for evaluating mild cognitive impairment and prodromal Alzheimer’s disease. Alzheimer’s Research & Therapy, 3, 35. Frank, L. et al. (2006a). Validation of a new symptom impact questionnaire for mild to moderate cognitive impairment. International Psychogeriatrics, 18, 135–149. Frank, L. et al. (2006b). Impact of cognitive impairment on mild dementia patients and mild cognitive impairment patients and their informants. International Psychogeriatrics, 18, 151–162. Gandek, B. et al. (1998). Cross-validation of item selection and scoring for the SF-12 Health Survey in nine countries: results from the IQOLA Project. International Quality of Life Assessment. Journal of Clinical Epidemiology, 51, 1171–1178. Ganguli, M. et al. (2011). Outcomes of mild cognitive impairment by definition: a population study. Archives of Neurology, 68, 761–767. Gauthier, S. et al. (2006). Mild cognitive impairment. Lancet, 367, 1262–1270. Jenkinson, C. et al. (2012). The development and validation of a quality of life measure for the carers of people with Parkinson’s disease (the PDQ-Carer). Parkinsonism & Related Disorders, 18, 483–487. Jones, G., Kennedy, S., Barnard, A., Wong, J. and Jenkinson, C. (2001). Development of an endometriosis quality-of-life instrument: the Endometriosis Health Profile-30. Obstetrics & Gynecology, 98, 258–264. Joosten-Weyn Banningh, L. W., Prins, J. B., Vernooij-Dassen, M. J., Wijnen, H. H., Olde Rikkert, M. G. and Kessels, R. P. (2011). Group therapy for patients with mild cognitive impairment and their significant others: results of a waiting-list controlled trial. Gerontology, 57, 444–454. Joosten-Weyn Banningh, L., Vernooij-Dassen, M., Rikkert, M. O. and Teunisse, J. P. (2008). Mild cognitive impairment: coping with an uncertain label. International Journal Geriatric Psychiatry, 23, 148–154. Kline, P. (1999). Handbook of Psychological Testing. London: Routledge. Lautenschlager, N. T., Cox, K. and Kurz, A. F. (2010). Physical activity and mild cognitive impairment and Alzheimer’s disease. Current Neurology and Neuroscience Reports, 10, 352–358. Lee, C. E., Browell, L. M. and Jones, D. L. (2008). Measuring health in patients with cervical and lumbosacral spinal disorders: is the 12-item short-form health survey a

A patient-reported outcome measure for MCI valid alternative for the 36-item short-form health survey? Archives of Physical Medicine and Rehabilitation, 89, 829–833. Likert, R. (1932). A technique for the measurements of attitudes. Archives of Psychology, 22, 1–55. Lu, Y.-F. Y., Haase, J. E. and Farran, C. J. (2007). Perspectives of persons with mild cognitive impairment: sense of being able. Alzheimer’s Care Today, 8, 75– 86. Lyketsos, C. G., Lopez, O., Jones, B., Fitzpatrick, A. L., Breitner, J. and DeKosky, S. (2002). Prevalence of neuropsychiatric symptoms in dementia and mild cognitive impairment: results from the cardiovascular health study. JAMA, 288, 1475–1483. Martinez, F. J. et al. (2008). Development and initial validation of a self-scored COPD Population Screener Questionnaire (COPD-PS). COPD, 5, 85–95. Matthews, F. E., Stephan, B. C., McKeith, I. G., Bond, J. and Brayne, C. (2008). Two-year progression from mild cognitive impairment to dementia: to what extent do different definitions agree? Journal of the American Geriatrics Society, 56, 1424–1433. Mitchell, A. J. (2008). The clinical significance of subjective memory complaints in the diagnosis of mild cognitive impairment and dementia: a meta-analysis. International Journal of Geriatric Psychiatry, 23, 1191–1202. National Institute for Clinical Excellence and Social Care Institute for Excellence (England and Wales) (2006). Dementia: Supporting People with Dementia and their Carers in Health and Social Care. London: National Institute for health and Clinical Excellence. Novella, J. L., Jochum, C., Ankri, J., Morrone, I., Jolly, D. and Blanchard, F. (2001). Measuring general health status in dementia: practical and methodological issues in using the SF-36. Aging, 13, 362–369. Nunnally, J. and Bernstein, I. (1994). Psychometric Theory. New York: McGraw Hill. World Health Organization. (2008). ICD-10: International Statistical Classification of Diseases and Related Health Problems. New York: World Health Organization. Palmer, K. and Fratiglioni, L. (2006). Is mild cognitive impairment a distinct clinical entity? Aging Health, 2, 763–769. Petersen, R. C., Smith, G. E., Waring, S. C., Ivnik, R. J., Tangalos, E. G. and Kokmen, E. (1999). Mild cognitive impairment: clinical characterization and outcome. Archives of Neurology, 56, 303–308. Portet, F. et al. (2006). Mild cognitive impairment (MCI) in medical practice: a critical review of the concept and new diagnostic procedure. Report of the MCI Working Group of the European Consortium on Alzheimer’s Disease.

497

Journal of Neurology, Neurosurgery, and Psychiatry, 77, 714–718. Roberts, J. L. and Clare, L. (2013). Meta-representational awareness in mild cognitive impairment: an interpretative phenomenological analysis. Aging & Mental Health, 17, 300–309. Salyers, M. P., Bosworth, H. B., Swanson, J. W., Lamb-Pagone, J. and Osher, F. C. (2000). Reliability and validity of the SF-12 health survey among people with severe mental illness. Medical Care, 38, 1141–1150. Schmid, R., Eschen, A., Ruegger-Frey, B. and Martin, M. (2012). Instruments for comprehensive needs assessment in individuals with cognitive complaints, mild cognitive impairment or dementia: a systematic review. International Journal of Geriatric Psychiatry, 27, 329–341. Smith, A. D. et al. (2010). Homocysteine-lowering by B vitamins slows the rate of accelerated brain atrophy in mild cognitive impairment: a randomized controlled trial. Public Library of Science, 5, e12244. Streiner, D. L. and Norman, G. R. (2003). Health Measurement Scales: A Practical Guide to their Development and Use. Oxford: Oxford University Press. US Department of Health and Human Services, Food and Drug Administration, Center for Drug Evaluation and Research and Center for Devices and Radiological Health (2009). Guidance for Industry. Patient-Reported Outcome Measures: Use in Medical Product Development to Support Labelling Claims. Maryland: FDA. Van der Mussele, S. et al. (2013). Behavioral symptoms in mild cognitive impairment as compared with Alzheimer’s disease and healthy older adults. International Journal of Geriatric Psychiatry, 28, 265–275. Vogel, A., Stokholm, J., Gade, A., Andersen, B. B., Hejl, A. M. and Waldemar, G. (2004). Awareness of deficits in mild cognitive impairment and Alzheimer’s disease: do MCI patients have impaired insight? Dementia and Geriatric Cognitive Disorders, 17, 181–187. Wahlund, L. O., Pihlstrand, E. and Jonhagen, M. E. (2003). Mild cognitive impairment: experience from a memory clinic. Acta Neurologica Scandinavica, 179, 21–24. Ware, J., Kosinski, M. and Keller, S. (1996). A 12-item short-form health survey: construction of scales and preliminary tests of reliability and validity. Medical Care, 34, 220–233. Ware, J., Kosinski, M., Turner-Bowker, D. and Gandek, B. (2007). User’s Manual for the SF-12v2 Health Survey. Lincoln, RI: QualityMetric Incorporated and Health Assessment Lab. Whitehouse, P. J. (2007). Mild cognitive impairment – a confused concept? Nature Clinical Practice Neurology, 3, 62–63.