International Journal of Psychiatry in Clinical Practice, 2007; 11(1): 36 43
ORIGINAL ARTICLE
Psychometric properties of the BASIS-24# (Behaviour and Symptom Identification Scale Revised) Mental Health Outcome Measure
I.M. CAMERON 1, L. CUNNINGHAM1, J.R. CRAWFORD2, J.M. EAGLES3, S.V. EISEN4, K. LAWTON5, S.A. NAJI6 & R.J. HAMILTON1 1
Department of Mental Health, University of Aberdeen, Aberdeen, UK, 2School of Psychology, University of Aberdeen, Aberdeen, UK, 3Royal Cornhill Hospital, Aberdeen, UK, 4Centre for Health Quality, Outcomes and Economic Research, Edith Nourse Rogers Memorial Veterans Hospital, Bedford, MA and Boston University School of Public Health, Boston, USA, 5 Department of General Practice and Primary Care, University of Aberdeen, Aberdeen, UK, and 6School of Health Sciences, Faculty of Health and Social Care, The Robert Gordon University, Aberdeen, UK
Abstract Objective. Outcome measurement in mental health services is an area of considerable clinical interest and policy priority. This study sought to assess the Behaviour and Symptom Identification Scale-24 (BASIS-24#), a brief, patient self-reported measure of psychopathology and functioning, in a UK sample, including establishing population norms for comparative purposes. Methods. Participants were 588 adults recruited from psychiatric inpatient, outpatient and primary care settings; and 630 adults randomly sampled from primary care lists who completed the BASIS-24#, and the Brief Symptom Inventory (BSI) at two time points. Results. BASIS-24# demonstrated adequate reliability (coefficient a values for combined clinical sample across subscales ranged from 0.75 to 0.91), validity and responsiveness to change (effect size for change of the BASIS-24# was 0.56 compared with 0.48 for BSI Global Severity Index). Population norms were established for the general population and adult in-patients (at in-take). The scale proved straightforward to complete across clinical settings. Variable rates of questionnaire distribution across clinical settings highlighted the ongoing challenge of incorporating outcome measures in clinical settings. Conclusion. BASIS-24# is a brief, easily administered, self-complete measure of mental well-being and functioning that adequately meets the requirements of reliability, validity and responsiveness to change required of an outcome measure.
Key Words: Outcome measurement, psychiatry, mental health
Introduction In the UK, as with elsewhere, mental health policy has highlighted the need to introduce routine outcome measurement into mental health care settings in order to provide useful information on patient progress at individual, local and national levels [1,2]. The Health of the Nation Outcome Scale (HoNOS) [3] a clinician-rated scale, has been identified and developed as the instrument of choice for this. Recognition has also been made for the need for patient-rated outcome measures to accompany clinician-rated measures [4]. The present authors propose the US developed BASIS-24# (Behaviour and Symptom Identification Scale-24): a simple, brief, self-complete measure of psychopathology and functioning for the routine collection of baseline and outcome data in mental health [5]. Preliminary
investigations identified its predecessor, the BASIS32† [6] as a promising instrument. This study sought to address the following research questions: 1) Does the BASIS-24# demonstrate psychometric robustness in a UK sample? 2) Can population norms be established for BASIS-24# which can provide useful comparison benchmarks for monitoring clinical progress?
Methods Samples Adult patients (age 18 65 years), with a new episode of a mental health problem were recruited from
Correspondence: Isobel Cameron, Research Fellow, Department of Mental Health, Institute of Medical Sciences, Foresterhill, Aberdeen AB25 2ZD, UK. Tel: /44 1224 557254. Fax: /44 1224 559320. E-mail: [email protected]
(Received 1 March 2006; accepted 15 June 2006) ISSN 1365-1501 print/ISSN 1471-1788 online # 2007 Taylor & Francis DOI: 10.1080/13651500600885531
Psychometric properties of the BASIS-24# three health care settings: psychiatric in-patient, Community Mental Health Teams (CMHTs) and primary care. Measures The BASIS-24# is a self-report questionnaire designed to measure outcome of mental health treatment from the service user’s viewpoint. Responses fall within six symptom and functioning domains: Depression/functioning, Interpersonal relationships, Psychotic symptoms, Alcohol/drug use, Emotional lability and Self-harm. Its predecessor, the BASIS32, whilst exhibiting face validity, ease of use and sensitivity to change following treatment, also had some limitations in terms of the reliability of some of its subscales, its between group differentiation and its accessibility to responders with limited literacy skills. For these reasons the present, revised version was developed. It can be viewed at www. basissurvey.org. The Brief Symptom Inventory (BSI) is a 53-tem measure of psychometric symptomatology [7]. By asking respondents to complete both measures, the responsiveness to change of the BASIS-24# could be considered in the context of the established BSI. Data collection All participants were asked to complete the BASIS24# and the BSI on two occasions: at the outset of the intervention and 3 months following (clinical sample), and at two time points 3 months apart (general population sample). In-patients met with a study researcher to receive the information before completing the questionnaires independently. CMHT participants received the study information from their treating clinician following which they completed the questionnaire at home and returned it by post. The primary care clinical sample received the study information by post from their general practitioner (primary care physician) shortly after attending the practice. The general population sample received their questionnaire by post. Samples recruited by post were provided with contact details of the research team, should they wish to discuss the study before consenting. Questionnaires for the second time point were distributed by post to all samples. For the clinical sample, diagnoses at time of recruitment were obtained from medical notes (or the Continuous Morbidity Recording Register in the case of primary care). Demographic information was collected at the outset. For the 3-month follow up, up to two reminders were sent.
37
population sample was analysed for responsiveness to change, though no change was expected in this group. The internal consistency of the scale was examined using Cronbach’s a in order to gauge the extent to which responses were consistent to items in the total scale, and items purporting to measure a subscale. Values of a falling between 0.7 and 0.9 were considered indicators of adequate internal consistency [8]. How well the scores of the BASIS-24# can predict external criteria was assessed by considering health care setting as an indicator of severity of symptoms reported. Data were analysed to assess whether responders in the in-patient setting reported greater problems and symptom distress than responders in the CMHT setting, and whether responders in the CMHT setting reported greater distress than responders in the primary care setting. The validity of the scale was further assessed by analysing whether the subscales differentiated patients in associated diagnostic groups. Responsiveness to clinical change over the two time points was measured by running paired t -tests on the BASIS-24# total score and the BSI Global Severity Index (GSI); the effect size of both measures was then calculated [9]. Where data were normally distributed t -tests were used to test for differences in group means and h2 used as a measure of effect size. Where scores were not normally distributed, Mann Whitney or Wilcoxon signed rank tests were carried out on these data. Analyses were carried out using SPSS. Power calculation Samples of 500 in both patient and general population groups at time point one are sufficient to yield reliability estimates with confidence intervals of 0.1 at the appropriate level of significance [10] and to generate stable percentiles for normative purposes [8]. The sample size also yields 90% power at the 5% significance level to assess the scale’s responsiveness to change (minimum detectable effect size of 0.12), and a mean difference of 0.18 between the groups. Ethical considerations Informed written consent was obtained before patients participated. This research was conducted with the approval of the Grampian Research Ethics Committee.
Results Analyses
Recruitment
All analyses were carried out on the clinical samples (in-patient, CMHT and primary care). The general
A sample of 588 patients was recruited from the three health care settings: psychiatric in-patient (n /331;
38
I.M. Cameron et al.
63% of patients approached), CMHTs (n /165; 37% of patients approached) and primary care (n / 92; 31% of patients approached). Figure 1 details how the in-patient sample was derived. In the CMHT sample 156 (26% of newly referred patients) did not receive packs where clinicians did not, or found it inappropriate to distribute the study information. In the primary care sample 41% of eligible patients did not receive study information where doctors felt it inappropriate (37%) or for other unspecified reasons (4%). A general population (i.e. non-clinical) sample of 630 (42% of 1513 adults (1865 years) randomly selected from three general practice lists) participated. The follow up questionnaire was completed by 418 (71%) of participants in the clinical arm (in-patient /219 (66%), CMHT /124 (75%) and primary care /75 (82%)) and 506 (80%) of participants in the general population sample. Sample characteristics Sample characteristics of all participants are shown in Table I. As might be expected, differences were observed between the groups in terms of demographic characteristics (educational qualifications, living arrangements and employment status) and primary diagnosis. Participants with a schizophrenic illness or bipolar affective disorder are almost entirely represented in the in-patient setting and predictably, almost the entire primary care sample had been consulting their General Practitioner about a depressive or anxiety disorder. The BASIS-24# raw item scores range from 0 (no difficulty/symptoms never present) to 4 (extreme difficulty/symptoms always present). Six items require reverse scoring. Following this the scale can be scored by calculating a mean score for each subscale and for the total scale. Mean scores have been found to be highly correlated with weighted mean scores developed more recently, and currently recommended for the BASIS-24# instrument [11]. Cases
which had more than six item responses missing were excluded. As the scores of the subscales were not normally distributed, medians are represented in Table II. Observational comparisons of scores can be made between samples, as well as between time points, both of which are subjected to statistical analyses further on in the results (see sections Concurrent criterion validity and Responsiveness to change). Reliability internal consistency Time point one a coefficients for the six subscales and total score from the clinical samples are shown in Table III. Coefficient a values for the total scale are robust and comparable across the clinical settings and time points. Values of a are also adequate for subscales across clinical groups with the exception of the psychosis and interpersonal relationships subscales in the primary care setting. Overall, reliability is higher in the in-patient and CMHT samples. Concurrent criterion validity Patients in the in-patient sample scored higher than patients in the CMHT sample; mean /2.04, sd / 0.69 versus 1.49, sd /0.62, p B/0.001, df /488, h2 /0.132. Patients in the CMHT sample scored higher than patients in the primary care sample; mean /1.49, sd /0.62 versus 1.18, sd /0.52, p B/ 0.001, df /254, h2 /0.061. The depression/functioning subscale did not distinguish the sample with a diagnosis of anxiety/depression from other patient samples; median /2.6 (IQR /1.5, 3.3) versus 2.83 (IQR /2, 3.5), p /0.09. The substance misuse scale successfully distinguished the substance misuse sample; median /2.5 (IQR /1.75, 3.25) versus 0.25 (IQR /0, 1), p B/0.001. The psychosis subscale successfully distinguished the psychotic sample; median /1.5 (IQR /0.75, 2.5) versus 0.75 (IQR / 0, 1.5), p B/0.001.
psychiatric admissions n=966
Eligible n=808
Ineligible n=158 (146 readmitted previous recruits; 12 not in age range)
Approached n=528
Not approached n=280
331 (63%) participated
115 too ill; 128 missed; 10 too hostile; 16 transferred; 11 other e.g. staff, spoke no English
Figure 1. In-patient sample recruitment.
Psychometric properties of the BASIS-24#
39
Table I. Sample characteristics. Characteristic N (%) unless stated otherwise Sex (male) Age (median, IQR*)
In-patient N /331 185 (56) 41 (30, 50)
CMHT N /165 66 (40)
27 (29)
37 (26, 48)
43 (35, 51)
Ethnicity White Asian Black Other
321 2 3 1
First language English
308 (93)
Education No qualification O level/GCSE/CSE/Scottish A level/Higher Teaching/HND/nursing Degree/post grad degree
91 104 38 39 41
(29) (33) (12) (13) (13)
25 44 38 27 28
Marital status Married/living with partner Separated Divorced Widowed Never married
95 51 55 11 108
(30) (16) (17) (3) (34)
76 16 13 4 54
Employment status Paid employment Volunteer worker Student Receiving benefits None For medical reasons For psychiatric reasons For substance misuse reasons Carstairs Deprivation Category (median, IQR*) Primary diagnosis Schizophrenia, schizo-affective disorder and other non affective psychotic disorders Bipolar affective disorder hypomanic phase Depressive/anxiety disorder Substance use disorder Other Nil psychiatric
(98) (B/1) (1) (B/1)
94 (30) 42 (13) 31 (9) 162 82 99 11
(49) (25) (30) (3)
4 (2, 4)
Primary care N /92
General population N /630 300 (48) 45 (34, 54)
163 (99) 2 (1) 0 0
91 (100) 0 0 0
610 11 7 1
(97) (2) (1) (B/1)
158 (96)
87 (95)
593 (96)
(15) (27) (24) (17) (17)
13 32 16 14 14
(15) (36) (18) (16) (16)
111 179 55 74 204
(18) (29) (9) (12) (33)
(47) (10) (8) (2) (33)
56 7 12 5 12
(61) (8) (13) (5) (13)
438 22 39 16 108
(70) (4) (6) (3) (17)
79 (48) 16 (10) 30 (19)
58 (64) 9 (10) 10 (11)
484 (77) 70 (11) 54 (9)
136 (82) 21 (13) 7 (4) 0
79 (86) 8 (9) 4 (4) 0
586 36 4 1
3 (2, 4)
2 (1, 4)
(94) (6) (B/1) (B/1)
2 (1,4) not applicable
93 (28) 36 149 32 15 5
(11) (45) (10) (5) (2)
5 (3) 1 141 3 5 10
(B/1) (85) (2) (3) (6)
0 0 90 (99) 1 (1) 0 0
* Interquartile range.
Responsiveness to change Paired t -tests on the combined clinical samples indicated a significant change from time point one to time point two in both the BASIS-24# and BSI, reflecting a reduction in psychopathology. The total score of the BASIS-24 was 1.68 (sd /0.71) at time point one, and 1.28 (sd /0.74) at time point two (p B/0.001), df /405. The BSI Global Severity Index (GSI) was 1.6 (sd /0.85) at time point one, and 1.19 (0.9) at time point two (p B/0.001), df / 397. The effect size for change of the BASIS-24# was 0.56 compared with 0.48 for the BSI GSI, indicating that the BASIS-24# is slightly more responsive to change than the 53-item BSI. Total scores in the general population sample were positively skewed, therefore BASIS-24# total scores at the two time points were subjected to Wilcoxon signed rank test. At time point one the median score
was 0.54 (0.29, 0.87) and at time point two 0.46 (0.25, 0.75), p /0.29. Similarly there was no difference between the BSI GSI scores at the two time points. At time point one the median score was 0.21 (0.08, 0.49) and at time point two 0.17 (0.06, 0.43), p /0.34. In a non-clinical, non-intervention sample, no significant difference was to be expected. Percentile tables To provide normative data for the BASIS-24#, a percentile table was constructed based on overall mean scores of the general population sample and the in-patient sample (Table IV). To illustrate the use of the percentile tables, suppose an individual’s score on the BASIS-24# is 2.00 on admission. By referring to Table IV, the percentiles derived from the general population sample show that the indivi-
1.92) 0.50) 0.50) 1.80) 2.33) 0.00) 1.25)
0.67 0.00 0.00 0.60 1.33 0.00 0.54
(0.17, (0.00, (0.00, (0.20, (0.67, (0.00, (0.29,
1.17) 0.50) 0.25) 1.20) 2.00) 0.00) 0.87)
0.50 0.00 0.00 0.60 1.00 0.00 0.46
(0.16, (0.00, (0.00, (0.20, (0.33, (0.00, (0.25,
1..00) 0.25) 0.25) 1..20) 1.67) 0.00) 0.75)
dual is at the 97th percentile; that is, their score is worse than 97% of the general population. Comparing this same raw score to the in-patient sample, it can be seen that this score is at the 62nd percentile; i.e. such a score is not unusual for inpatient admissions but is towards the more severe end. Suppose also that the individual’s BASIS-24# score at follow-up was 0.67. It can be seen that, in the intervening period, the patient’s disturbance has become much less marked; they are now broadly scoring in the normal range although still above the average of the general population (the score is at the 61st percentile).
(0.58, (0.00, (0.00, (0.40, (1.00, (0.00, (0.46,
Discussion
3.00) 1.00) 1.19) 2.40) 2.00) 1.00) 1.96)
1.67 0.00 0.50 1.80 2.00 0.00 1.21
(1.00, (0.00, (0.00, (0.95, (1.33, (0.00, (0.77,
2.71) 0.75) 1.00) 2.40) 2.67) 1.00) 1.82)
2.00 0.00 0.25 1.40 1.33 0.00 1.19
(1.33, (0.00, (0.00, (0.80, (1.00, (0.00, (0.76,
2.67) 0.50) 0.50) 1.80) 2.00) 0.50) 1.49)
1.33 0.00 0.00 1.00 1.67 0.00 0.88
BASIS-24# is a brief, easily administered, selfcomplete measure of mental well-being and functioning that adequately meets the requirements of reliability, validity and responsiveness to change required of an outcome measure. This version of the BASIS has demonstrated improved psychometric properties in relation to its predecessor the BASIS-32† [6] which while demonstrating strengths also exhibited marginal internal consistency and poor between group differentiation in some of its subscales. Additionally, the current psychometric investigations are of greater relevance across the spectrum of mental health service provision, with inclusion of patients from diverse clinical settings. The previous UK validation of the BASIS-32 was based on an in-patient sample only [6].
2.00 0.25 0.50 1.78 2.00 0.50 1.34
(1.00, (0.00, (0.00, (1.00, (1.00, (0.00, (0.79,
3.00) 1.25) 1.25) 2.58) 2.33) 1.50) 1.94)
2.33 0.25 0.50 1.80 1.33 0.00 1.45
(1.50, (0.00, (0.00, (1.20, (1.00, (0.00, (1.04,
Representativeness of sample
* Interquartile range.
3.00 (2.33, 3.67) 0.75 (0.00, 2.25) 1.25 (0.50, 2.25) 2.20 (1.45, 3.00) 2.17 (1.33, 2.67) 1.5 (0.50, 3.00) 2.08 (1.62, 2.51)
Time point 1 N /92 Time point 1 N /164 Time point 1 N /326 BASIS-24 scale
Depression/functioning Substance misuse Psychosis Interpersonal relationships Emotional lability Self-harm Total scale
Time point 1 N /625
Follow-up (3 months) N /505 Time /95 days (92, 107) Follow-up (3 months) N /74 Time /97 days (92, 106) Follow-up (3 months) N /122 Time /99 days (92, 106) Follow-up (3 months) N /216 Time /96 days (92,106)
In-patient sample Median (IQR*)
CMHT sample Median (IQR)
Primary Care sample Median (IQR)
General Population sample Median (IQR)
I.M. Cameron et al.
Table II. Median scores of BASIS-24.
40
Sample characteristics identified patients with a broad range of diagnoses in proportions which correspond to the clinical setting of presentation. The lack of ethnic diversity is reflective of Scotland generally [12]. The range of deprivation categories is representative of the Grampian area which reflects less poverty than Scotland as a whole [13]. As noted earlier, there were some large, expected differences in demographic characteristics between the samples. The sample was sought from in-patient, CMHT and primary care in order to assess the scale’s utility across a range of clinical settings. The in-patient setting was well represented, the CMHT adequately so, but less so was the primary care setting. In this setting a high proportion of eligible patients (41%) did not receive study information from participating general practitioners. It was evident that some general practitioners were reluctant to make requests of patients to participate in the research perhaps because they perceived that such a request could have brought additional stress to an already burdened patient group. This highlights the continuing challenge of implementing an outcome assessment instrument in clinical settings.
Psychometric properties of the BASIS-24#
41
Table III. Internal consistency (a values) clinical at time point 1. BASIS-24 Scale
Depression/functioning Substance misuse Psychosis Interpersonal relationships Emotional lability Self-harm Total scale
In-patient (min n /258) Coefficient a (95% CI*) 0.88 0.85 0.76 0.78 0.70 0.91 0.88
(0.85, (0.82, (0.72, (0.74, (0.64, (0.89, (0.86,
0.90) 0.88) 0.80) 0.82) 0.75) 0.93) 0.90)
CMHT (min n /146) Coefficient a (95% CI) 0.90 0.80 0.70 0.77 0.73 0.83 0.90
(0.88, (0.75, (0.62, (0.71, (0.65, (0.76, (0.88,
0.93) 0.85) 0.77) 0.82) 0.79) 0.87) 0.92)
Primary care (min n /80) All Clinical (min n /484) Coefficient a (95% CI) Coefficient a (95% CI) 0.87 0.68 0.57 0.52 0.71 0.85 0.84
(0.83, (0.55, (0.41, (0.33, (0.59, (0.77, (0.79,
0.91) 0.78) 0.70) 0.66) 0.80) 0.90) 0.89)
0.90 0.85 0.79 0.75 0.71 0.91 0.91
(0.88. (0.82, (0.76, (0.72, (0.67, (0.90, (0.89,
0.91) 0.87) 0.81) 0.79) 0.75) 0.93) 0.92)
*CI, confidence interval.
An important requirement of a self-complete measure is that it should be widely acceptable to a broad range of patients. In the most acute setting the scale was found manageable by most patients with severe and enduring mental illness. In only 12% of admissions did clinicians feel their patients were too ill to participate. This proportion may become less in time as clinicians become more familiar with the scale and its general acceptability increases. Inevitably, as with all psychiatric self-complete scales, there will be a proportion of patients who will be too ill to manage to complete such a task. The lower baseline response rate in the CMHT and primary care settings is not thought to relate to ease of completion but may be explained by the method of distribution. In these settings potential participants had the questionnaire to consider at home. Greater motivation would be required to complete and return it than in the in-patient setting where the researchers visited the wards daily to distribute and collect the study packs. Psychometric robustness in a research, audit or clinical setting The total scale showed good internal consistency across all the clinical samples. Within subscales, a values for psychosis and interpersonal relationships were less robust in the primary care setting. This is attributable to lesser variability in the primary care sample than in the inpatient or CMHT samples. As high a values may suggest some redundancy among items, reduction in the length of the total scale might be possible without compromising reliability. However, it was only in the combined sample of the total scale and the Self-harm subscale that a values exceeded 0.9. As the Self-harm subscale consists of two items, having a minimum of two items per subscale was viewed as preferable to having a single item subscale, despite some redundancy. The BASIS-24# also demonstrated good concurrent criterion validity. The finding that the ‘depression/functioning’ subscale did not differentiate by diagnosis is explicable in terms of symptoms of anxiety and depression being widely experienced across other diagnoses.
Responsiveness to change was also demonstrated showing that the BASIS-24# could be useful as a tool for monitoring patient progress over time and for assessing interventions in a research context. Utility Although outcome measurement in mental health services has emerged as an area of considerable interest and priority [1,2,14,15], there has been little evidence of their use [16]. This has partly been explained by concerns expressed relating to (a) the basic psychometric properties of available measures, (b) questions relating to the perceived usefulness of outcome data to clinical practice and (c) concerns relating to inadequacy of infrastructures that would allow for systematic data collection and useful employment. As the BASIS-24# has been demonstrated to be psychometrically robust, this first concern is adequately addressed. The second concern partly reflects the culture of wariness surrounding the use of outcome measures [17,18]. The utility of the information collected in the BASIS-24# will best be assessed by evaluating how successfully the measure can be introduced into clinical services and subsequently assessing whether it provides useful feedback to clinicians and managers. The utility of feedback to clinicians and managers is an important consideration. It has been noted that outcome measures are of limited use to clinicians when they are not available to them while they are making clinical decisions about patients [19]. The method of scoring BASIS-24# by calculating a mean may have some advantages over the endorsed weighted algorithm method which requires computer input before a score is calculated. Whilst this is acceptable for assessing aggregated clinical data, it is impractical for the clinician at ground level who is interested in change scores of individual patients. Internet-based, automated scoring has recently been developed to address this need [20]. Simple scoring methods in outcome measurement are necessary for their usefulness to clinicians. In the case of BASIS-24# it also allows for the practical use of the developed percentile tables which set a context for considering individual patient’s scores.
42
I.M. Cameron et al.
Table IV. BASIS-24 mean overall score percentiles for in-patient and general population samples. Percentile 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50
In-patient sample
General population sample
Percentile
In-patient sample
General population sample
0.29 0.36 0.41 0.50 0.54 0.57 0.65 0.70 0.74 0.75 0.79 0.83 0.88 0.92 0.96 0.96 1.00 1.04 1.08 1.13 1.13 1.17 1.21 1.21 1.25 1.25 1.27 1.29 1.30 1.33 1.35 1.41 1.42 1.43 1.45 1.46 1.48 1.50 1.52 1.54 1.57 1.58 1.61 1.61 1.63 1.65 1.67 1.71 1.74 1.74
0.00 0.04 0.04 0.04 0.08 0.08 0.10 0.13 0.13 0.17 0.17 0.17 0.17 0.21 0.21 0.21 0.21 0.25 0.25 0.25 0.25 0.29 0.29 0.29 0.29 0.30 0.30 0.30 0.30 0.33 0.33 0.36 0.36 0.38 0.38 0.42 0.42 0.42 0.43 0.43 0.43 0.46 0.46 0.48 0.48 0.48 0.50 0.52 0.52 0.52
51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 /99
1.75 1.78 1.79 1.82 1.83 1.88 1.91 1.92 1.96 1.96 2.00 2.00 2.04 2.04 2.08 2.08 2.13 2.14 2.17 2.17 2.21 2.21 2.22 2.25 2.27 2.29 2.30 2.33 2.38 2.39 2.43 2.46 2.48 2.50 2.54 2.60 2.63 2.68 2.74 2.75 2.82 2.86 2.89 2.92 3.00 3.09 3.17 3.25 3.30 3.46
0.54 0.58 0.58 0.58 0.63 0.63 0.63 0.63 0.67 0.67 0.67 0.71 0.71 0.71 0.74 0.74 0.75 0.75 0.78 0.78 0.79 0.83 0.83 0.87 0.87 0.88 0.91 0.92 0.96 1.00 1.00 1.04 1.04 1.08 1.09 1.13 1.17 1.21 1.25 1.29 1.33 1.33 1.46 1.54 1.63 1.75 2.00 2.17 2.50 2.88
Note: where a raw score corresponds to more than one percentile take the higher percentile.
The third concern, that of resources, is an important consideration. The challenges inherent in introducing routine outcome measurement into clinical practice were particularly highlighted in the data collection for the primary care sample. It may be that where outcome measures are introduced for routine audit purposes there will be less obstacles present than are necessary in the conduct of research as patients will have less to complete in routine practice where concurrent measures are not used. Also, in research, potential participants have to consider giving informed written consent before
completing the questionnaire. Additionally, current ethics policy dictates that individuals have up to 24 h to consider whether to participate or not. In the case of routine audit, written consent is not required and perhaps therefore participation appears less daunting than in the research context. Conversely, in some settings the research process may have enhanced participation. In the in-patient setting, a researcher was present daily to facilitate involvement. In Grampian, BASIS-24# has recently been introduced as part of the clinical audit of a new urgent referral service. The practical application of the
43
Psychometric properties of the BASIS-24# measure will be observed with keen interest. If mental health services are serious about the implementation of routine outcome measures, protected time to assess and use data is also required.
[2] Secretary of State for Health. National Service Framework Mental Health. London: HSMO; 1999. [3] Wing JK, Beevor AS, Curtis RH, Park SB, Hadden S, Burns A. Health of the Nation Outcome Scales (HoNOS). Br J Psychiatry 1998;172:11 8. [4] Eagar K, Trauer T, Mellsop G. Performance of routine outcome measurement in adult mental health care. Aust NZ J Psychiatry 2005;39:713 8. [5] Eisen SV, Normand SLT, Belanger AJ, Gevorkian S, Irvin EA. BASIS 32† and the Revised Behavior and Symptom Identification Scale (BASIS). In: Maruish M, editor. The use of psychological testing for treatment planning and outcome assessment, 3rd ed. NJ: Lawrence Erlbaum Associates; 2004. p 79 115. [6] Cameron IM, Eagles JM, Howie FL, Andrew JE, Crawford JR, Kohler C, Eisen S, et al. Preliminary validation of a UKmodified version of the BASIS-32. Int J Psychiatry Clin Pract 2001;5:41 8. [7] Derogatis LR. The Brief Symptom Inventory: Administration, scoring and procedures. Minneapolis, MN: National Computer Systems; 1993. [8] Nunally JC, Bernstein IH. Psychometric theory, 3rd ed. New York: McGraw-Hill; 1994. [9] Deyo RA, Diehr P, Patrick DL. Reproducibility and responsiveness of health status measures: statistics and strategies for evaluation. Control Clin Trials 1991;12:142S 58S. [10] Streiner DL, Norman GR. Health measurement scales: a practical guide to their development and use. Oxford: Oxford University Press; 1989. [11] Eisen SV, personal communication, BASIS-24 Instruction Guide, McLean Hospital, Department of Mental Health Services Evaluation, 2005. [12] Office of the Chief Statistician. Analysis of Ethnicity in the 2001 Census: Summary Report http://www.scotland.gov.uk/ library5/social/aescr-00.asp, 2004. [13] Scottish Health Statistics http://www.show.scot.nhs.uk/is donline/, 2000. [14] Gilbody SM, House AO, Sheldon TA. Outcomes research in mental health. Br J Psychiatry 2002;181:8 16. [15] Holloway F. Outcome measurement in mental health welcome to the revolution. Br J Psychiatry 2002;181:1 2. [16] Gilbody SM, House AO, Sheldon TA. Psychiatrists in the UK do not use outcome measures. Br J Psychiatry 2002;180: 101 3. [17] Ashaye OA, Livingston G, Orrell MW. Does stadardized needs assessment improve the outcome of psychiatric day hospital care for older people? A randomized controlled trial. Aging Ment Health 2003;7:195 9. [18] Marshall M, Lockwood A, Green G, Zajac-Roles G, Roberts C, Harrison G. Systematic assessments of need and care planning in severe mental illness; Cluster randomised trial. Br J Psychiatry 2004;185:163 8. [19] Andrews G, Page AC. Outcome measurement, outcome management and monitoring. Aust NZ J Psychiatry 2005; 39:649 51. [20] http://basissurvey.org/webscore24/ /
Key points
/
/
. The BASIS-24# is a brief, simple to complete self-report measure of psychopathology and functioning suitable for routine clinical use . It has demonstrated reliability, validity and responsiveness to change in a diverse clinical sample and can be considered robust . The BASIS-24# is not appropriate for use with a minority of acutely ill patients for whom a selfcomplete measure is not feasible . It provides valuable information from the patient perspective . Clinician perspectives should also be considered to form a more complete picture Acknowledgements This research was funded by a grant from the Chief Scientist’s Office of the Scottish Executive (Reference number CZH/4/39) and by the US Department of Veterans Affairs, Veterans Health Administration, Health Services Research and Development Service. The views expressed in this article are those of the authors and do not necessarily represent the views of the Scottish Executive or the Department of Veterans Affairs. Administrative and clinical workers at Royal Cornhill Hospital and at five General Practices in Grampian are gratefully acknowledged for facilitating with recruitment.
/
/
/
/
/
/
/
/
/
/
Statement of interest BASIS-24# is copyrighted and licensed for use by McLean Hospital. As part of the hospital’s intellectual property policy, the instrument developer, Susan V Eisen, receives a percentage of licensing fees received by McLean Hospital for licensing of the instrument.
/
/
References
/
/
/
/
[1] Scottish Office. A framework for mental health services in Scotland. Edinburgh: Scottish Office; 1997.
/
ORIGINAL ARTICLE
Psychometric properties of the BASIS-24# (Behaviour and Symptom Identification Scale Revised) Mental Health Outcome Measure
I.M. CAMERON 1, L. CUNNINGHAM1, J.R. CRAWFORD2, J.M. EAGLES3, S.V. EISEN4, K. LAWTON5, S.A. NAJI6 & R.J. HAMILTON1 1
Department of Mental Health, University of Aberdeen, Aberdeen, UK, 2School of Psychology, University of Aberdeen, Aberdeen, UK, 3Royal Cornhill Hospital, Aberdeen, UK, 4Centre for Health Quality, Outcomes and Economic Research, Edith Nourse Rogers Memorial Veterans Hospital, Bedford, MA and Boston University School of Public Health, Boston, USA, 5 Department of General Practice and Primary Care, University of Aberdeen, Aberdeen, UK, and 6School of Health Sciences, Faculty of Health and Social Care, The Robert Gordon University, Aberdeen, UK
Abstract Objective. Outcome measurement in mental health services is an area of considerable clinical interest and policy priority. This study sought to assess the Behaviour and Symptom Identification Scale-24 (BASIS-24#), a brief, patient self-reported measure of psychopathology and functioning, in a UK sample, including establishing population norms for comparative purposes. Methods. Participants were 588 adults recruited from psychiatric inpatient, outpatient and primary care settings; and 630 adults randomly sampled from primary care lists who completed the BASIS-24#, and the Brief Symptom Inventory (BSI) at two time points. Results. BASIS-24# demonstrated adequate reliability (coefficient a values for combined clinical sample across subscales ranged from 0.75 to 0.91), validity and responsiveness to change (effect size for change of the BASIS-24# was 0.56 compared with 0.48 for BSI Global Severity Index). Population norms were established for the general population and adult in-patients (at in-take). The scale proved straightforward to complete across clinical settings. Variable rates of questionnaire distribution across clinical settings highlighted the ongoing challenge of incorporating outcome measures in clinical settings. Conclusion. BASIS-24# is a brief, easily administered, self-complete measure of mental well-being and functioning that adequately meets the requirements of reliability, validity and responsiveness to change required of an outcome measure.
Key Words: Outcome measurement, psychiatry, mental health
Introduction In the UK, as with elsewhere, mental health policy has highlighted the need to introduce routine outcome measurement into mental health care settings in order to provide useful information on patient progress at individual, local and national levels [1,2]. The Health of the Nation Outcome Scale (HoNOS) [3] a clinician-rated scale, has been identified and developed as the instrument of choice for this. Recognition has also been made for the need for patient-rated outcome measures to accompany clinician-rated measures [4]. The present authors propose the US developed BASIS-24# (Behaviour and Symptom Identification Scale-24): a simple, brief, self-complete measure of psychopathology and functioning for the routine collection of baseline and outcome data in mental health [5]. Preliminary
investigations identified its predecessor, the BASIS32† [6] as a promising instrument. This study sought to address the following research questions: 1) Does the BASIS-24# demonstrate psychometric robustness in a UK sample? 2) Can population norms be established for BASIS-24# which can provide useful comparison benchmarks for monitoring clinical progress?
Methods Samples Adult patients (age 18 65 years), with a new episode of a mental health problem were recruited from
Correspondence: Isobel Cameron, Research Fellow, Department of Mental Health, Institute of Medical Sciences, Foresterhill, Aberdeen AB25 2ZD, UK. Tel: /44 1224 557254. Fax: /44 1224 559320. E-mail: [email protected]
(Received 1 March 2006; accepted 15 June 2006) ISSN 1365-1501 print/ISSN 1471-1788 online # 2007 Taylor & Francis DOI: 10.1080/13651500600885531
Psychometric properties of the BASIS-24# three health care settings: psychiatric in-patient, Community Mental Health Teams (CMHTs) and primary care. Measures The BASIS-24# is a self-report questionnaire designed to measure outcome of mental health treatment from the service user’s viewpoint. Responses fall within six symptom and functioning domains: Depression/functioning, Interpersonal relationships, Psychotic symptoms, Alcohol/drug use, Emotional lability and Self-harm. Its predecessor, the BASIS32, whilst exhibiting face validity, ease of use and sensitivity to change following treatment, also had some limitations in terms of the reliability of some of its subscales, its between group differentiation and its accessibility to responders with limited literacy skills. For these reasons the present, revised version was developed. It can be viewed at www. basissurvey.org. The Brief Symptom Inventory (BSI) is a 53-tem measure of psychometric symptomatology [7]. By asking respondents to complete both measures, the responsiveness to change of the BASIS-24# could be considered in the context of the established BSI. Data collection All participants were asked to complete the BASIS24# and the BSI on two occasions: at the outset of the intervention and 3 months following (clinical sample), and at two time points 3 months apart (general population sample). In-patients met with a study researcher to receive the information before completing the questionnaires independently. CMHT participants received the study information from their treating clinician following which they completed the questionnaire at home and returned it by post. The primary care clinical sample received the study information by post from their general practitioner (primary care physician) shortly after attending the practice. The general population sample received their questionnaire by post. Samples recruited by post were provided with contact details of the research team, should they wish to discuss the study before consenting. Questionnaires for the second time point were distributed by post to all samples. For the clinical sample, diagnoses at time of recruitment were obtained from medical notes (or the Continuous Morbidity Recording Register in the case of primary care). Demographic information was collected at the outset. For the 3-month follow up, up to two reminders were sent.
37
population sample was analysed for responsiveness to change, though no change was expected in this group. The internal consistency of the scale was examined using Cronbach’s a in order to gauge the extent to which responses were consistent to items in the total scale, and items purporting to measure a subscale. Values of a falling between 0.7 and 0.9 were considered indicators of adequate internal consistency [8]. How well the scores of the BASIS-24# can predict external criteria was assessed by considering health care setting as an indicator of severity of symptoms reported. Data were analysed to assess whether responders in the in-patient setting reported greater problems and symptom distress than responders in the CMHT setting, and whether responders in the CMHT setting reported greater distress than responders in the primary care setting. The validity of the scale was further assessed by analysing whether the subscales differentiated patients in associated diagnostic groups. Responsiveness to clinical change over the two time points was measured by running paired t -tests on the BASIS-24# total score and the BSI Global Severity Index (GSI); the effect size of both measures was then calculated [9]. Where data were normally distributed t -tests were used to test for differences in group means and h2 used as a measure of effect size. Where scores were not normally distributed, Mann Whitney or Wilcoxon signed rank tests were carried out on these data. Analyses were carried out using SPSS. Power calculation Samples of 500 in both patient and general population groups at time point one are sufficient to yield reliability estimates with confidence intervals of 0.1 at the appropriate level of significance [10] and to generate stable percentiles for normative purposes [8]. The sample size also yields 90% power at the 5% significance level to assess the scale’s responsiveness to change (minimum detectable effect size of 0.12), and a mean difference of 0.18 between the groups. Ethical considerations Informed written consent was obtained before patients participated. This research was conducted with the approval of the Grampian Research Ethics Committee.
Results Analyses
Recruitment
All analyses were carried out on the clinical samples (in-patient, CMHT and primary care). The general
A sample of 588 patients was recruited from the three health care settings: psychiatric in-patient (n /331;
38
I.M. Cameron et al.
63% of patients approached), CMHTs (n /165; 37% of patients approached) and primary care (n / 92; 31% of patients approached). Figure 1 details how the in-patient sample was derived. In the CMHT sample 156 (26% of newly referred patients) did not receive packs where clinicians did not, or found it inappropriate to distribute the study information. In the primary care sample 41% of eligible patients did not receive study information where doctors felt it inappropriate (37%) or for other unspecified reasons (4%). A general population (i.e. non-clinical) sample of 630 (42% of 1513 adults (1865 years) randomly selected from three general practice lists) participated. The follow up questionnaire was completed by 418 (71%) of participants in the clinical arm (in-patient /219 (66%), CMHT /124 (75%) and primary care /75 (82%)) and 506 (80%) of participants in the general population sample. Sample characteristics Sample characteristics of all participants are shown in Table I. As might be expected, differences were observed between the groups in terms of demographic characteristics (educational qualifications, living arrangements and employment status) and primary diagnosis. Participants with a schizophrenic illness or bipolar affective disorder are almost entirely represented in the in-patient setting and predictably, almost the entire primary care sample had been consulting their General Practitioner about a depressive or anxiety disorder. The BASIS-24# raw item scores range from 0 (no difficulty/symptoms never present) to 4 (extreme difficulty/symptoms always present). Six items require reverse scoring. Following this the scale can be scored by calculating a mean score for each subscale and for the total scale. Mean scores have been found to be highly correlated with weighted mean scores developed more recently, and currently recommended for the BASIS-24# instrument [11]. Cases
which had more than six item responses missing were excluded. As the scores of the subscales were not normally distributed, medians are represented in Table II. Observational comparisons of scores can be made between samples, as well as between time points, both of which are subjected to statistical analyses further on in the results (see sections Concurrent criterion validity and Responsiveness to change). Reliability internal consistency Time point one a coefficients for the six subscales and total score from the clinical samples are shown in Table III. Coefficient a values for the total scale are robust and comparable across the clinical settings and time points. Values of a are also adequate for subscales across clinical groups with the exception of the psychosis and interpersonal relationships subscales in the primary care setting. Overall, reliability is higher in the in-patient and CMHT samples. Concurrent criterion validity Patients in the in-patient sample scored higher than patients in the CMHT sample; mean /2.04, sd / 0.69 versus 1.49, sd /0.62, p B/0.001, df /488, h2 /0.132. Patients in the CMHT sample scored higher than patients in the primary care sample; mean /1.49, sd /0.62 versus 1.18, sd /0.52, p B/ 0.001, df /254, h2 /0.061. The depression/functioning subscale did not distinguish the sample with a diagnosis of anxiety/depression from other patient samples; median /2.6 (IQR /1.5, 3.3) versus 2.83 (IQR /2, 3.5), p /0.09. The substance misuse scale successfully distinguished the substance misuse sample; median /2.5 (IQR /1.75, 3.25) versus 0.25 (IQR /0, 1), p B/0.001. The psychosis subscale successfully distinguished the psychotic sample; median /1.5 (IQR /0.75, 2.5) versus 0.75 (IQR / 0, 1.5), p B/0.001.
psychiatric admissions n=966
Eligible n=808
Ineligible n=158 (146 readmitted previous recruits; 12 not in age range)
Approached n=528
Not approached n=280
331 (63%) participated
115 too ill; 128 missed; 10 too hostile; 16 transferred; 11 other e.g. staff, spoke no English
Figure 1. In-patient sample recruitment.
Psychometric properties of the BASIS-24#
39
Table I. Sample characteristics. Characteristic N (%) unless stated otherwise Sex (male) Age (median, IQR*)
In-patient N /331 185 (56) 41 (30, 50)
CMHT N /165 66 (40)
27 (29)
37 (26, 48)
43 (35, 51)
Ethnicity White Asian Black Other
321 2 3 1
First language English
308 (93)
Education No qualification O level/GCSE/CSE/Scottish A level/Higher Teaching/HND/nursing Degree/post grad degree
91 104 38 39 41
(29) (33) (12) (13) (13)
25 44 38 27 28
Marital status Married/living with partner Separated Divorced Widowed Never married
95 51 55 11 108
(30) (16) (17) (3) (34)
76 16 13 4 54
Employment status Paid employment Volunteer worker Student Receiving benefits None For medical reasons For psychiatric reasons For substance misuse reasons Carstairs Deprivation Category (median, IQR*) Primary diagnosis Schizophrenia, schizo-affective disorder and other non affective psychotic disorders Bipolar affective disorder hypomanic phase Depressive/anxiety disorder Substance use disorder Other Nil psychiatric
(98) (B/1) (1) (B/1)
94 (30) 42 (13) 31 (9) 162 82 99 11
(49) (25) (30) (3)
4 (2, 4)
Primary care N /92
General population N /630 300 (48) 45 (34, 54)
163 (99) 2 (1) 0 0
91 (100) 0 0 0
610 11 7 1
(97) (2) (1) (B/1)
158 (96)
87 (95)
593 (96)
(15) (27) (24) (17) (17)
13 32 16 14 14
(15) (36) (18) (16) (16)
111 179 55 74 204
(18) (29) (9) (12) (33)
(47) (10) (8) (2) (33)
56 7 12 5 12
(61) (8) (13) (5) (13)
438 22 39 16 108
(70) (4) (6) (3) (17)
79 (48) 16 (10) 30 (19)
58 (64) 9 (10) 10 (11)
484 (77) 70 (11) 54 (9)
136 (82) 21 (13) 7 (4) 0
79 (86) 8 (9) 4 (4) 0
586 36 4 1
3 (2, 4)
2 (1, 4)
(94) (6) (B/1) (B/1)
2 (1,4) not applicable
93 (28) 36 149 32 15 5
(11) (45) (10) (5) (2)
5 (3) 1 141 3 5 10
(B/1) (85) (2) (3) (6)
0 0 90 (99) 1 (1) 0 0
* Interquartile range.
Responsiveness to change Paired t -tests on the combined clinical samples indicated a significant change from time point one to time point two in both the BASIS-24# and BSI, reflecting a reduction in psychopathology. The total score of the BASIS-24 was 1.68 (sd /0.71) at time point one, and 1.28 (sd /0.74) at time point two (p B/0.001), df /405. The BSI Global Severity Index (GSI) was 1.6 (sd /0.85) at time point one, and 1.19 (0.9) at time point two (p B/0.001), df / 397. The effect size for change of the BASIS-24# was 0.56 compared with 0.48 for the BSI GSI, indicating that the BASIS-24# is slightly more responsive to change than the 53-item BSI. Total scores in the general population sample were positively skewed, therefore BASIS-24# total scores at the two time points were subjected to Wilcoxon signed rank test. At time point one the median score
was 0.54 (0.29, 0.87) and at time point two 0.46 (0.25, 0.75), p /0.29. Similarly there was no difference between the BSI GSI scores at the two time points. At time point one the median score was 0.21 (0.08, 0.49) and at time point two 0.17 (0.06, 0.43), p /0.34. In a non-clinical, non-intervention sample, no significant difference was to be expected. Percentile tables To provide normative data for the BASIS-24#, a percentile table was constructed based on overall mean scores of the general population sample and the in-patient sample (Table IV). To illustrate the use of the percentile tables, suppose an individual’s score on the BASIS-24# is 2.00 on admission. By referring to Table IV, the percentiles derived from the general population sample show that the indivi-
1.92) 0.50) 0.50) 1.80) 2.33) 0.00) 1.25)
0.67 0.00 0.00 0.60 1.33 0.00 0.54
(0.17, (0.00, (0.00, (0.20, (0.67, (0.00, (0.29,
1.17) 0.50) 0.25) 1.20) 2.00) 0.00) 0.87)
0.50 0.00 0.00 0.60 1.00 0.00 0.46
(0.16, (0.00, (0.00, (0.20, (0.33, (0.00, (0.25,
1..00) 0.25) 0.25) 1..20) 1.67) 0.00) 0.75)
dual is at the 97th percentile; that is, their score is worse than 97% of the general population. Comparing this same raw score to the in-patient sample, it can be seen that this score is at the 62nd percentile; i.e. such a score is not unusual for inpatient admissions but is towards the more severe end. Suppose also that the individual’s BASIS-24# score at follow-up was 0.67. It can be seen that, in the intervening period, the patient’s disturbance has become much less marked; they are now broadly scoring in the normal range although still above the average of the general population (the score is at the 61st percentile).
(0.58, (0.00, (0.00, (0.40, (1.00, (0.00, (0.46,
Discussion
3.00) 1.00) 1.19) 2.40) 2.00) 1.00) 1.96)
1.67 0.00 0.50 1.80 2.00 0.00 1.21
(1.00, (0.00, (0.00, (0.95, (1.33, (0.00, (0.77,
2.71) 0.75) 1.00) 2.40) 2.67) 1.00) 1.82)
2.00 0.00 0.25 1.40 1.33 0.00 1.19
(1.33, (0.00, (0.00, (0.80, (1.00, (0.00, (0.76,
2.67) 0.50) 0.50) 1.80) 2.00) 0.50) 1.49)
1.33 0.00 0.00 1.00 1.67 0.00 0.88
BASIS-24# is a brief, easily administered, selfcomplete measure of mental well-being and functioning that adequately meets the requirements of reliability, validity and responsiveness to change required of an outcome measure. This version of the BASIS has demonstrated improved psychometric properties in relation to its predecessor the BASIS-32† [6] which while demonstrating strengths also exhibited marginal internal consistency and poor between group differentiation in some of its subscales. Additionally, the current psychometric investigations are of greater relevance across the spectrum of mental health service provision, with inclusion of patients from diverse clinical settings. The previous UK validation of the BASIS-32 was based on an in-patient sample only [6].
2.00 0.25 0.50 1.78 2.00 0.50 1.34
(1.00, (0.00, (0.00, (1.00, (1.00, (0.00, (0.79,
3.00) 1.25) 1.25) 2.58) 2.33) 1.50) 1.94)
2.33 0.25 0.50 1.80 1.33 0.00 1.45
(1.50, (0.00, (0.00, (1.20, (1.00, (0.00, (1.04,
Representativeness of sample
* Interquartile range.
3.00 (2.33, 3.67) 0.75 (0.00, 2.25) 1.25 (0.50, 2.25) 2.20 (1.45, 3.00) 2.17 (1.33, 2.67) 1.5 (0.50, 3.00) 2.08 (1.62, 2.51)
Time point 1 N /92 Time point 1 N /164 Time point 1 N /326 BASIS-24 scale
Depression/functioning Substance misuse Psychosis Interpersonal relationships Emotional lability Self-harm Total scale
Time point 1 N /625
Follow-up (3 months) N /505 Time /95 days (92, 107) Follow-up (3 months) N /74 Time /97 days (92, 106) Follow-up (3 months) N /122 Time /99 days (92, 106) Follow-up (3 months) N /216 Time /96 days (92,106)
In-patient sample Median (IQR*)
CMHT sample Median (IQR)
Primary Care sample Median (IQR)
General Population sample Median (IQR)
I.M. Cameron et al.
Table II. Median scores of BASIS-24.
40
Sample characteristics identified patients with a broad range of diagnoses in proportions which correspond to the clinical setting of presentation. The lack of ethnic diversity is reflective of Scotland generally [12]. The range of deprivation categories is representative of the Grampian area which reflects less poverty than Scotland as a whole [13]. As noted earlier, there were some large, expected differences in demographic characteristics between the samples. The sample was sought from in-patient, CMHT and primary care in order to assess the scale’s utility across a range of clinical settings. The in-patient setting was well represented, the CMHT adequately so, but less so was the primary care setting. In this setting a high proportion of eligible patients (41%) did not receive study information from participating general practitioners. It was evident that some general practitioners were reluctant to make requests of patients to participate in the research perhaps because they perceived that such a request could have brought additional stress to an already burdened patient group. This highlights the continuing challenge of implementing an outcome assessment instrument in clinical settings.
Psychometric properties of the BASIS-24#
41
Table III. Internal consistency (a values) clinical at time point 1. BASIS-24 Scale
Depression/functioning Substance misuse Psychosis Interpersonal relationships Emotional lability Self-harm Total scale
In-patient (min n /258) Coefficient a (95% CI*) 0.88 0.85 0.76 0.78 0.70 0.91 0.88
(0.85, (0.82, (0.72, (0.74, (0.64, (0.89, (0.86,
0.90) 0.88) 0.80) 0.82) 0.75) 0.93) 0.90)
CMHT (min n /146) Coefficient a (95% CI) 0.90 0.80 0.70 0.77 0.73 0.83 0.90
(0.88, (0.75, (0.62, (0.71, (0.65, (0.76, (0.88,
0.93) 0.85) 0.77) 0.82) 0.79) 0.87) 0.92)
Primary care (min n /80) All Clinical (min n /484) Coefficient a (95% CI) Coefficient a (95% CI) 0.87 0.68 0.57 0.52 0.71 0.85 0.84
(0.83, (0.55, (0.41, (0.33, (0.59, (0.77, (0.79,
0.91) 0.78) 0.70) 0.66) 0.80) 0.90) 0.89)
0.90 0.85 0.79 0.75 0.71 0.91 0.91
(0.88. (0.82, (0.76, (0.72, (0.67, (0.90, (0.89,
0.91) 0.87) 0.81) 0.79) 0.75) 0.93) 0.92)
*CI, confidence interval.
An important requirement of a self-complete measure is that it should be widely acceptable to a broad range of patients. In the most acute setting the scale was found manageable by most patients with severe and enduring mental illness. In only 12% of admissions did clinicians feel their patients were too ill to participate. This proportion may become less in time as clinicians become more familiar with the scale and its general acceptability increases. Inevitably, as with all psychiatric self-complete scales, there will be a proportion of patients who will be too ill to manage to complete such a task. The lower baseline response rate in the CMHT and primary care settings is not thought to relate to ease of completion but may be explained by the method of distribution. In these settings potential participants had the questionnaire to consider at home. Greater motivation would be required to complete and return it than in the in-patient setting where the researchers visited the wards daily to distribute and collect the study packs. Psychometric robustness in a research, audit or clinical setting The total scale showed good internal consistency across all the clinical samples. Within subscales, a values for psychosis and interpersonal relationships were less robust in the primary care setting. This is attributable to lesser variability in the primary care sample than in the inpatient or CMHT samples. As high a values may suggest some redundancy among items, reduction in the length of the total scale might be possible without compromising reliability. However, it was only in the combined sample of the total scale and the Self-harm subscale that a values exceeded 0.9. As the Self-harm subscale consists of two items, having a minimum of two items per subscale was viewed as preferable to having a single item subscale, despite some redundancy. The BASIS-24# also demonstrated good concurrent criterion validity. The finding that the ‘depression/functioning’ subscale did not differentiate by diagnosis is explicable in terms of symptoms of anxiety and depression being widely experienced across other diagnoses.
Responsiveness to change was also demonstrated showing that the BASIS-24# could be useful as a tool for monitoring patient progress over time and for assessing interventions in a research context. Utility Although outcome measurement in mental health services has emerged as an area of considerable interest and priority [1,2,14,15], there has been little evidence of their use [16]. This has partly been explained by concerns expressed relating to (a) the basic psychometric properties of available measures, (b) questions relating to the perceived usefulness of outcome data to clinical practice and (c) concerns relating to inadequacy of infrastructures that would allow for systematic data collection and useful employment. As the BASIS-24# has been demonstrated to be psychometrically robust, this first concern is adequately addressed. The second concern partly reflects the culture of wariness surrounding the use of outcome measures [17,18]. The utility of the information collected in the BASIS-24# will best be assessed by evaluating how successfully the measure can be introduced into clinical services and subsequently assessing whether it provides useful feedback to clinicians and managers. The utility of feedback to clinicians and managers is an important consideration. It has been noted that outcome measures are of limited use to clinicians when they are not available to them while they are making clinical decisions about patients [19]. The method of scoring BASIS-24# by calculating a mean may have some advantages over the endorsed weighted algorithm method which requires computer input before a score is calculated. Whilst this is acceptable for assessing aggregated clinical data, it is impractical for the clinician at ground level who is interested in change scores of individual patients. Internet-based, automated scoring has recently been developed to address this need [20]. Simple scoring methods in outcome measurement are necessary for their usefulness to clinicians. In the case of BASIS-24# it also allows for the practical use of the developed percentile tables which set a context for considering individual patient’s scores.
42
I.M. Cameron et al.
Table IV. BASIS-24 mean overall score percentiles for in-patient and general population samples. Percentile 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50
In-patient sample
General population sample
Percentile
In-patient sample
General population sample
0.29 0.36 0.41 0.50 0.54 0.57 0.65 0.70 0.74 0.75 0.79 0.83 0.88 0.92 0.96 0.96 1.00 1.04 1.08 1.13 1.13 1.17 1.21 1.21 1.25 1.25 1.27 1.29 1.30 1.33 1.35 1.41 1.42 1.43 1.45 1.46 1.48 1.50 1.52 1.54 1.57 1.58 1.61 1.61 1.63 1.65 1.67 1.71 1.74 1.74
0.00 0.04 0.04 0.04 0.08 0.08 0.10 0.13 0.13 0.17 0.17 0.17 0.17 0.21 0.21 0.21 0.21 0.25 0.25 0.25 0.25 0.29 0.29 0.29 0.29 0.30 0.30 0.30 0.30 0.33 0.33 0.36 0.36 0.38 0.38 0.42 0.42 0.42 0.43 0.43 0.43 0.46 0.46 0.48 0.48 0.48 0.50 0.52 0.52 0.52
51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 /99
1.75 1.78 1.79 1.82 1.83 1.88 1.91 1.92 1.96 1.96 2.00 2.00 2.04 2.04 2.08 2.08 2.13 2.14 2.17 2.17 2.21 2.21 2.22 2.25 2.27 2.29 2.30 2.33 2.38 2.39 2.43 2.46 2.48 2.50 2.54 2.60 2.63 2.68 2.74 2.75 2.82 2.86 2.89 2.92 3.00 3.09 3.17 3.25 3.30 3.46
0.54 0.58 0.58 0.58 0.63 0.63 0.63 0.63 0.67 0.67 0.67 0.71 0.71 0.71 0.74 0.74 0.75 0.75 0.78 0.78 0.79 0.83 0.83 0.87 0.87 0.88 0.91 0.92 0.96 1.00 1.00 1.04 1.04 1.08 1.09 1.13 1.17 1.21 1.25 1.29 1.33 1.33 1.46 1.54 1.63 1.75 2.00 2.17 2.50 2.88
Note: where a raw score corresponds to more than one percentile take the higher percentile.
The third concern, that of resources, is an important consideration. The challenges inherent in introducing routine outcome measurement into clinical practice were particularly highlighted in the data collection for the primary care sample. It may be that where outcome measures are introduced for routine audit purposes there will be less obstacles present than are necessary in the conduct of research as patients will have less to complete in routine practice where concurrent measures are not used. Also, in research, potential participants have to consider giving informed written consent before
completing the questionnaire. Additionally, current ethics policy dictates that individuals have up to 24 h to consider whether to participate or not. In the case of routine audit, written consent is not required and perhaps therefore participation appears less daunting than in the research context. Conversely, in some settings the research process may have enhanced participation. In the in-patient setting, a researcher was present daily to facilitate involvement. In Grampian, BASIS-24# has recently been introduced as part of the clinical audit of a new urgent referral service. The practical application of the
43
Psychometric properties of the BASIS-24# measure will be observed with keen interest. If mental health services are serious about the implementation of routine outcome measures, protected time to assess and use data is also required.
[2] Secretary of State for Health. National Service Framework Mental Health. London: HSMO; 1999. [3] Wing JK, Beevor AS, Curtis RH, Park SB, Hadden S, Burns A. Health of the Nation Outcome Scales (HoNOS). Br J Psychiatry 1998;172:11 8. [4] Eagar K, Trauer T, Mellsop G. Performance of routine outcome measurement in adult mental health care. Aust NZ J Psychiatry 2005;39:713 8. [5] Eisen SV, Normand SLT, Belanger AJ, Gevorkian S, Irvin EA. BASIS 32† and the Revised Behavior and Symptom Identification Scale (BASIS). In: Maruish M, editor. The use of psychological testing for treatment planning and outcome assessment, 3rd ed. NJ: Lawrence Erlbaum Associates; 2004. p 79 115. [6] Cameron IM, Eagles JM, Howie FL, Andrew JE, Crawford JR, Kohler C, Eisen S, et al. Preliminary validation of a UKmodified version of the BASIS-32. Int J Psychiatry Clin Pract 2001;5:41 8. [7] Derogatis LR. The Brief Symptom Inventory: Administration, scoring and procedures. Minneapolis, MN: National Computer Systems; 1993. [8] Nunally JC, Bernstein IH. Psychometric theory, 3rd ed. New York: McGraw-Hill; 1994. [9] Deyo RA, Diehr P, Patrick DL. Reproducibility and responsiveness of health status measures: statistics and strategies for evaluation. Control Clin Trials 1991;12:142S 58S. [10] Streiner DL, Norman GR. Health measurement scales: a practical guide to their development and use. Oxford: Oxford University Press; 1989. [11] Eisen SV, personal communication, BASIS-24 Instruction Guide, McLean Hospital, Department of Mental Health Services Evaluation, 2005. [12] Office of the Chief Statistician. Analysis of Ethnicity in the 2001 Census: Summary Report http://www.scotland.gov.uk/ library5/social/aescr-00.asp, 2004. [13] Scottish Health Statistics http://www.show.scot.nhs.uk/is donline/, 2000. [14] Gilbody SM, House AO, Sheldon TA. Outcomes research in mental health. Br J Psychiatry 2002;181:8 16. [15] Holloway F. Outcome measurement in mental health welcome to the revolution. Br J Psychiatry 2002;181:1 2. [16] Gilbody SM, House AO, Sheldon TA. Psychiatrists in the UK do not use outcome measures. Br J Psychiatry 2002;180: 101 3. [17] Ashaye OA, Livingston G, Orrell MW. Does stadardized needs assessment improve the outcome of psychiatric day hospital care for older people? A randomized controlled trial. Aging Ment Health 2003;7:195 9. [18] Marshall M, Lockwood A, Green G, Zajac-Roles G, Roberts C, Harrison G. Systematic assessments of need and care planning in severe mental illness; Cluster randomised trial. Br J Psychiatry 2004;185:163 8. [19] Andrews G, Page AC. Outcome measurement, outcome management and monitoring. Aust NZ J Psychiatry 2005; 39:649 51. [20] http://basissurvey.org/webscore24/ /
Key points
/
/
. The BASIS-24# is a brief, simple to complete self-report measure of psychopathology and functioning suitable for routine clinical use . It has demonstrated reliability, validity and responsiveness to change in a diverse clinical sample and can be considered robust . The BASIS-24# is not appropriate for use with a minority of acutely ill patients for whom a selfcomplete measure is not feasible . It provides valuable information from the patient perspective . Clinician perspectives should also be considered to form a more complete picture Acknowledgements This research was funded by a grant from the Chief Scientist’s Office of the Scottish Executive (Reference number CZH/4/39) and by the US Department of Veterans Affairs, Veterans Health Administration, Health Services Research and Development Service. The views expressed in this article are those of the authors and do not necessarily represent the views of the Scottish Executive or the Department of Veterans Affairs. Administrative and clinical workers at Royal Cornhill Hospital and at five General Practices in Grampian are gratefully acknowledged for facilitating with recruitment.
/
/
/
/
/
/
/
/
/
/
Statement of interest BASIS-24# is copyrighted and licensed for use by McLean Hospital. As part of the hospital’s intellectual property policy, the instrument developer, Susan V Eisen, receives a percentage of licensing fees received by McLean Hospital for licensing of the instrument.
/
/
References
/
/
/
/
[1] Scottish Office. A framework for mental health services in Scotland. Edinburgh: Scottish Office; 1997.
/
