557502 research-article2014
HPQ0010.1177/1359105314557502Journal of Health PsychologyLeck et al.
Article Journal of Health Psychology 1–10 © The Author(s) 2014 Reprints and permissions: sagepub.co.uk/journalsPermissions.nav DOI: 10.1177/1359105314557502 hpq.sagepub.com
Social Return on Investment: Valuing health outcomes or promoting economic values? Chris Leck1, Dominic Upton2 and Nick Evans1
Abstract Interventions and activities that influence health are often concerned with intangible outcomes that are difficult to value despite their potential significance. Social Return on Investment is an evaluation framework that explores all aspects of change and expresses these in comparable terms. It combines qualitative narratives and quantitative measurements with a financial approach to enable outcomes that can otherwise be overlooked or undervalued to be incorporated appropriately. This article presents Social Return on Investment as an effective tool for supporting the development of a holistic appreciation of how interventions impact on the health and well-being of individuals, communities and societies.
Keywords cost-effectiveness, health psychology, outcomes, values, well-being
Introduction Interventions and activities that have relevance to health psychology can be multifactorial and concerned with intangible outcomes that are challenging to identify and measure, let alone to conceptualise meaningfully (Fitzpatrick, 2000; O’Carroll, 2014; Powdthavee and Van den Berg, 2011; Stamuli, 2011). This can result in elements of associated value (negative and positive) being overlooked, misunderstood or inappropriately assigned, despite their perhaps having significance in relation to individual and collective lives (Nicholls et al., 2012; Owens et al., 2011). As Owens et al. (2011) observed, ‘judgments about value are fundamental to decision making in most arenas but often have been missing in health care settings’ (p. 174). Health psychology needs to be able to identify and articulate the changes in behaviour and
circumstances that occur as a result of interventions and activities (Fitzpatrick, 2000; O’Carroll, 2014). Numerous studies have indicated over at least the last four decades that health and wellbeing are influenced by a combination of social, environmental and economic factors (Suls and Rothman, 2004). Therefore, research tools are required that can incorporate consideration of that which has relevance to all such discourses rather than focusing solely on those aspects of change that are pre-defined or immediately 1University 2University
of Worcester, UK of Canberra, Australia
Corresponding author: Chris Leck, University of Worcester, Bredon 65, Henwick Grove, Worcester WR2 6AJ, UK. Email: [email protected]
Downloaded from hpq.sagepub.com at UNIVERSITE LAVAL on November 14, 2015
2
Journal of Health Psychology
quantifiable (Marks, 1996; Schmidt et al., 2011). ‘In short, health psychologists need to design research that embraces rather than shies away from the complexity of the phenomena of interest’ (Suls and Rothman, 2004: 123). Social Return on Investment (SROI) is an outcome-focused framework that seeks to enable such research and is being applied increasingly for evaluation purposes (Arvidson et al., 2010; Ebrahim and Rangan, 2010). It combines qualitative narratives and quantitative measurements with a financial approach to enable associated change to be identified, explored, conceptualised and valued in comparable terms (Arvidson et al., 2013; Nicholls et al., 2012). SROI is intended to serve the needs of those who access, provide, fund and evaluate services and interventions (Arvidson et al., 2010; Harlock, 2013; HM Government, 2010; Martin et al., 2010; Nicholls et al., 2012; Wood and Leighton, 2010). SROI analyses seek to take account of all social, environmental and economic outcomes – the ‘triple bottom line’ (Elkington, 1997) – and articulate their associated value in monetary terms (Arvidson et al., 2010; Harlock, 2013; Nicholls et al., 2012). A return on investment ratio is calculated, and this is the element that is often highlighted, but SROI is intended to be about more than generating an economic interpretation of value (Arvidson et al., 2013; Nicholls et al., 2012). Quantitative data and qualitative explorations of real-world contexts and experiences are combined to provide an enhanced understanding of reality (Dures et al., 2011; Nicholls et al., 2012). SROI supports a biopsychosocial approach to health as it allows biological and psychological factors to be considered in conjunction with the relevance of the social and physical world that is inhabited (Engel, 1977, 1980; Sarafino, 2006). A large volume of grey literature detailing the application of SROI in the health and social care sector is now available, but it is an impact measurement methodology that has not previously received scrutiny with regard to its relevance and applicability for the purposes of health psychology. This reality needs to be
addressed in recognition of the fact that, ‘as the use of economic evaluation studies continues to grow, it is important that patients, clinicians and decision makers, as well as the general public, understand the key value judgements which lie behind these assessments’ (Stamuli, 2011: 208). This article supports the development of such an understanding by describing the principles and process that underpin SROI. It outlines techniques that are applied to measure and quantify the sort of intangible or ‘soft’ outcomes (such as improved mood or enhanced social inclusion) that are difficult to value but contribute towards health and well-being. The strengths and challenges associated with the methodology are discussed to assess its suitability as a framework to support the requirements of health psychology. The article references a case study to help illustrate how SROI has been applied practically to support the evaluation of a healthrelated intervention (Leck, 2012). The study under consideration relates to a UK care farm and has been assured by the SROI Network. Many definitions of care farming are available, reflecting specific national policy contexts, but it concerns vulnerable people being enabled to engage with farming activities to promote their health and well-being (Leck et al., 2014). As the descriptor for the activity suggests, the provision of care is an integral element, but this is not unidirectional. The intent is to enable participants to care for themselves and one another in addition to the livestock, crops and wider ‘natural’ environment (Leck et al., 2014). Participants are encouraged to learn new skills and engage in productive work within an intentionally inclusive and therapeutic place.
Applying SROI to understand impact The approach to SROI that informs this article is outlined in ‘A guide to social return on investment’ (Nicholls et al., 2012). This publication was originally funded by the UK Government Cabinet Office in 2009, but has since been updated. The following principles and stages
Downloaded from hpq.sagepub.com at UNIVERSITE LAVAL on November 14, 2015
3
Leck et al. are highlighted in both editions as fundamental to the overall process: SROI principles: 1. 2. 3. 4. 5. 6. 7.
Involve material stakeholders Understand what changes Value what matters Include only what is material Avoid over-claiming Be transparent Verify the result
SROI process: 1. 2. 3. 4. 5. 6.
Establish scope and key stakeholders Map outcomes Evidence and value outcomes Establish impact Calculate the SROI Report, use and embed
Stakeholder identification and involvement are incorporated as the first stage and principle of SROI. The term ‘stakeholder’ refers to all those who are identified as potentially experiencing change because of the activities under consideration. These might, for example, include individuals who are directly involved (such as service users and volunteers), members of personal support networks (such as family and carers), organisations (such as hospitals and social care providers) and wider society (as a result, for example, of reduced benefit payments or increased social cohesion). The scope of an SROI is clarified at the outset and identifies the activities to be included, the period of time to which it relates and the type of analysis to be applied (evaluation or forecast). All potentially relevant stakeholder groups are identified and their materiality for the purposes of the SROI is assessed according to the form and extent of anticipated change (Nicholls et al., 2012). However, it is always possible that significant stakeholder groups will be overlooked or undervalued during the initial stages of an evaluation. SROI requires that relevance is continuously reassessed to ensure that
all those who experience change can be taken into consideration (Arvidson et al., 2010; Nicholls et al., 2012). The development of an impact map is a central feature of SROI. This articulates a theory of change and encompasses the data that inform the calculation of the return on investment ratio (Nicholls et al., 2012). Theory of change is a research approach that sits well within the SROI framework because it links activities to outcomes and supports the development of a qualitative awareness of relevant relationships from the outset (Kail and Lumley, 2012). Inputs and outputs that relate to all material stakeholder groups are initially identified and recorded on the impact map (Nicholls et al., 2012). While the former encompass that which is invested (e.g. salaries paid), the latter incorporate information about that which is provided (e.g. number of sessions delivered). This is generally a fairly straightforward part of the process, with informing quantitative data being provided by practitioners and service providers. The identification of outcomes, and the selection of indicators and proxies that conceptualise these, is more complex. Processes and outcomes are often explored initially through qualitative consultations with stakeholders, but quantitative data pertaining to appropriate indicators evidence the extent to which they apply and ascertain associated value (Nicholls et al., 2012). If, for example, an outcome relates to increased self-confidence, then an appropriate indicator might concern someone engaging in new social activities. An over-reliance on selfreported (subjective) measures can be avoided by also obtaining supporting quantitative data and evidence from more objective third-party sources. Outcomes are valued through the application of justifiable proxies that have monetary value (Arvidson et al., 2010; Nicholls et al., 2012). These can be identified from a variety of external sources (such as databases, academic analyses and other SROI studies), but it is imperative that they are recognised by those to whom they apply as being appropriate and realistic economic valuations of the change that they experience.
Downloaded from hpq.sagepub.com at UNIVERSITE LAVAL on November 14, 2015
4
Journal of Health Psychology
The proxies that are applied can only ever be surrogates, and some variation will always exist between individuals, but this element of the process is intended to enable outcomes that might otherwise be overlooked to be afforded appropriate recognition (Nicholls et al., 2012). Crucially, it can allow ‘intangible’ outcomes (alternatively described as ‘soft’ or ‘indirect’), which do not have direct economic value, but can be critical to health and well-being, to be explored and incorporated in the analysis (Arvidson et al., 2010, 2013; Diener and Chan, 2011; Wood and Leighton, 2010). Outcomes concern the changes that occur during the period of time under consideration, but impact relates to the contribution that the intervention itself makes to this change (Steed and Nicholles, 2011). This distinction is easily overlooked when interventions are being evaluated and can result in a distorted picture being presented. SROI incorporates procedures intended specifically to allow external influences to be taken into account and excluded (Arvidson et al., 2013; Nicholls et al., 2012). These identify that which would have happened regardless of the intervention taking place (deadweight), has occurred as a result of other input (attribution) or that has moved rather than being removed (displacement). Attribution can be particularly relevant in relation to complex health and social care issues that often involve input from multiple agencies (Steed and Nicholles, 2011). The figures provided can only ever be estimates, but they are, once again, underpinned by evidence provided by the stakeholders concerned. Additionally, SROI acknowledges that the effects of some outcomes last longer than others. While some are dependent on the activity continuing, others might continue to exert influence, albeit of a diminishing form, into the future. In such instances, and assuming sufficient evidence is available to adequately justify such a claim, duration can be extended beyond the time period under consideration and the value applied to the related change increased accordingly. All outcome values (positive and negative) that are attributable to the activity under
consideration are added together on the impact map and compared to the original investment to provide the SROI ratio (Nicholls et al., 2012). Sensitivity analysis is subsequently applied to test the overall effect on the final return ratio of particular outcomes/proxies and associated estimates/assumptions. Findings are then shared with stakeholders to ensure that they perceive that which is reported as accurately reflecting their own reality. Consideration will now be given to the practical application of SROI. Associated strengths and challenges are explored in order to assess the extent to which it is a framework that has applicability in social and health-care settings. Specific elements of the previously referenced care farm SROI are incorporated to illustrate that which is described.
Material stakeholders It is a central, distinctive and defining feature of SROI that consideration is applied to all material stakeholders (Arvidson et al., 2010; Nicholls et al., 2012). This helps ensure that all related outcomes are valued in the analysis rather than solely those that apply directly to the primary intended beneficiary. For example, data collected for the care farm SROI identified significant associated value for the National Health Service (NHS) and the carers and family members of the service users. This latter group indicated that relationships had improved, and that they had benefited from having time to relax and recuperate, comfortable in the knowledge that the person they usually took responsibility for supporting was happy, safe and actively engaged elsewhere. Most care farm service users had previously required in-patient hospital treatment in connection with their condition, but only one individual had needed to be readmitted since attending the farm (Leck, 2012). Additionally, the care farm was found to enable physical exercise and participating in food production (from field to fork) was presented as encouraging people to adopt a healthy diet and lifestyle. Given that body mass index (BMI)–related illnesses have
Downloaded from hpq.sagepub.com at UNIVERSITE LAVAL on November 14, 2015
5
Leck et al. been calculated to cost the NHS 17.4 billion pounds sterling per annum (McPherson et al., 2011), associated savings were judged to be significant and worthy of inclusion. Various other changes that will result similarly in cost savings at a national level were also identified in the care farm case study. These included reduced benefit payments, drug/alcohol use, social/educational support needs and public disorder/crime. However, they were not found to be sufficiently widespread to justify inclusion on the impact map. This will have resulted in the final SROI ratio being lower than might otherwise have been the case, and the significance of such change for the individuals concerned still needed to be highlighted, but it is not practical to incorporate outcomes on a case-by-case basis. Only 61 per cent of the total value associated with the care farm was found ultimately to relate to the service users themselves. Another 18 per cent concerned their carers/relatives and 17 per cent took the form of savings to the NHS resulting from reduced use of services (Leck, 2012). This latter outcome is unlikely to translate into direct and immediate financial savings, but value is derived from resources being freed for application elsewhere. It is a particular strength of SROI, and something that is often absent in relation to project evaluations, that it can provide such an understanding of wherein value actually lies.
Understanding outcomes SROI is a framework that can support health researchers in describing and quantifying the wider value that interventions can provide for individuals, communities and society. A holistic approach is adopted that allows a meaningful understanding of what is really happening to be provided rather than exploring solely that which is pre-defined or anticipated (Nicholls et al., 2012). SROI embraces consideration of outcomes that are not immediately quantifiable rather than focusing exclusively on those that are more easily measured.
Intentionally incorporated flexibility helps ensure that it is not only intended outcomes that are identified but also those that are unintended and can be overlooked if performance is evaluated solely according to preordained objectives and measures (Nicholls et al., 2012). All stakeholders are directly involved throughout the process and unanticipated or unintended outcomes can be explored and incorporated effectively in conjunction with that which has been pre-defined. Multiple outcomes that influence health and well-being will often be cited or observed during an evaluation process and this has the potential to result in excessive claims being applied if all are valued independently. With regard to the care farm SROI, relevant outcomes were found to include increased levels of relaxation, happiness, satisfaction, confidence, social engagement and feelings of self-worth. However, it would have been neither realistic nor appropriate to value each of these outcomes separately. This would have increased the return on investment ratio, but it would also have resulted effectively in the same changes being counted on more than one occasion. SROI manages this risk by incorporating such related changes within chains of events rather than as outcomes to be valued individually (Nicholls et al., 2012). The following chains were developed in conjunction with the care farm service users to accommodate the key outcomes that were found to apply among this group of stakeholders: Service users got structure to their day → learnt new skills → did something productive in the workplace (meaningful activity) → became more confident → received job satisfaction. Service users enjoyed being in a restorative natural environment → became more relaxed → felt happier → interacted with others → made friends → became part of a supportive and reciprocal social network. Service users were active on the farm → adopted a healthier lifestyle and diet → physical health improved.
Downloaded from hpq.sagepub.com at UNIVERSITE LAVAL on November 14, 2015
6
Journal of Health Psychology
Financial proxies were only assigned on the care farm impact map to the most advanced stage of these chains of events (Nicholls et al., 2012). This process allowed all relevant outcomes to be evidenced and incorporated into the wider analysis while also reducing the risk of an inaccurate picture being provided.
Assigning proxy values The assigning of monetary values to outcomes is an element of SROI that is both fundamental and contentious, and no consensus exists regarding the most appropriate techniques (Donovan, 2013; Fujiwara and Campbell, 2011; Powdthavee and Van den Berg, 2011). Contingent valuation methods have been applied fairly widely, with SROI analyses having utilised both ‘revealed preference’ and ‘stated preference’ techniques (Fujiwara and Campbell, 2011; HM Government, 2010). Revealed preference techniques observe how people behave in related markets (hedonic pricing) or calculate costs associated with accessing services, but these can be challenging to apply in relation to the sort of intangible outcomes that have relevance to health and well-being (Fujiwara and Campbell, 2011; Powdthavee and Van den Berg, 2011). Some stated preference techniques involve value being assigned via the concepts of ‘willingness to pay’ for an outcome to happen or ‘willingness to accept’ in terms of the compensation required to replace the outcome (Fujiwara and Campbell, 2011; Powdthavee and Van den Berg, 2011). Choice modelling is another stated preference strategy, that allows respondents to be provided with various alternatives that they can then rank (Donovan, 2013). Associated exercises can incorporate the outcome concerned alongside other changes that have more apparent financial value. These might, for example, include things such as a new pair of shoes, a new car or a holiday. The ordering of responses can help conceptualise the relative value attributed to the outcome concerned by the person to whom it applies.
Another method for valuing intangible outcomes that influence health and well-being is to apply statistical analyses to large national data sets (Fujiwara, 2013; Powdthavee and Van den Berg, 2011). Measured effects, relating for example to happiness or quality of life, are on this occasion compared with the impact associated with additional income through the application of regression analysis (Arvidson et al., 2013; Fujiwara and Campbell, 2011). This technique has been presented as more accurately reflecting personal realities as opposed to that which is imagined to apply (Fujiwara and Campbell, 2011). However, valuations for the same health condition can vary fairly substantially according to the well-being measure that is applied (Powdthavee and Van den Berg, 2011). With regard to the care farm case study, paid employment in a competitive marketplace was not currently a realistic option for many participants. Nevertheless, and as indicated previously, those concerned obtained job satisfaction from their active participation on the farm. A 1-point rise in job satisfaction (on a 10-point scale) has been calculated as equating to 36 per cent of income (Helliwell and Huang, 2005), and this figure was on this occasion applied to the minimum UK wage to obtain an appropriate proxy value. Given the fact that proxy values can be obtained via a range of methods, it is essential that those that are applied are recognised as appropriate by those to whom they apply. Becoming part of a genuine and effective social network was also valued for the purposes of the care farm SROI according to data obtained from the application of statistical analysis to a large national data set, but the stakeholders concerned did not on this occasion feel that associated figures represented their reality. A shadow pricing method that was applied to the British Household Panel Survey had presented increased social involvement as worth the equivalent of up to £85,000 per annum, with a more modest figure of £15,500 being suggested to reflect the value of seeing friends or relatives on most days of the week rather than one or two (Powdthavee, 2008). However, even this lower
Downloaded from hpq.sagepub.com at UNIVERSITE LAVAL on November 14, 2015
7
Leck et al. figure was considered excessive by care farm participants. Discussions with those concerned identified 25 per cent of this latter figure as more closely equating to their reality. Conceptualising less tangible outcomes in economic terms will always incorporate challenges and be dependent to some extent on personally informed value judgments, but this aspect has been suggested additionally to distract attention from the evidence that underpins the analysis and increase the likelihood of research being used inappropriately (Arvidson et al., 2010; Gibbon and Dey, 2011; Harlock, 2013). Consideration is now given to the validity of such claims.
Measuring value The SROI Guide emphasises that the ratio of return on investment should not be the sole reason for engaging with SROI, considered in isolation or compared between organisations (Arvidson et al., 2010, 2013; Lyon et al., 2010; Nicholls et al., 2012). However, current austerity measures and competitive realities in the United Kingdom and elsewhere are contributing towards health and social care policy-makers and funders focusing increasingly on the maximisation of the return that their investment provides (Arvidson et al., 2013). This is suggested to increase the risk of undue emphasis being placed on the ratio, comparisons being drawn and attempts being made to inflate or hide relevant aspects (Arvidson et al., 2013; Gibbon and Dey, 2011; Milbourne, 2013). Such realities are undeniable, and debates regarding the application of financial indicators to outcomes that do not have a direct monetary value will continue, but the relevance of market-led economic processes in the health sector has to be acknowledged (Gibbon and Dey, 2011). ‘The economic evaluation of healthcare interventions has become an integral part of health policy and decision-making for the allocation of the limited healthcare resources’ (Stamuli, 2011: 197). Incorporated proxies cannot be presented as, or claimed to be, precisely
applicable in all instances – the extent of associated change and the degree to which it is valued will always vary between individuals – but their inclusion helps ensure that the value of that which is not immediately quantifiable, but is nevertheless relevant, can be afforded appropriate recognition. It is understandable that particular attention is given to the ratio of return because it is a distinctive feature of SROI, allows total associated value to be more easily conceptualised and is a useful peg on which to hang an exploration of impact. However, this number is not intended to be considered in isolation (Nicholls et al., 2012). The reductive and quantitative element provided by the ratio should be applied and interpreted in conjunction with the summative assessment that is more qualitatively informed. The intent is not simply to express associated value numerically but rather to provide a holistic understanding of that which takes place, associated processes and relative import. The presentation or perception of SROI as a market-based evaluation tool concerned solely with monetary value will not sit comfortably with those who engage with health and social interventions that are not for profit or have philanthropic intent (Arvidson et al., 2013). Nevertheless, it is naive and unrealistic to imagine that economic realities do not apply to health and social care (Owens et al., 2011). The UK government is a significant funder in the sector, promotes competitive tendering and requires impact to be demonstrable (HM Government, 2011, 2012, 2013; The National Council for Voluntary Organisations, 2011). Equating SROI with the marketisation of the subject of the evaluation is failing to understand its fundamental purpose. It is intended to be about value rather than money and to measure value rather than merely value measures (Luke et al., 2013; Nicholls et al., 2012). SROI seeks to enable all that which takes place as a result of an activity to be taken into consideration rather than solely that which has direct applicability in the marketplace. The purpose is to articulate the current value while also providing an understanding of the sources of that value.
Downloaded from hpq.sagepub.com at UNIVERSITE LAVAL on November 14, 2015
8
Journal of Health Psychology
In many ways, the elements of SROI that present the greatest challenges are those that can be perceived also as providing its strengths. It is a framework that seeks to financially quantify all elements of value rather than solely those that are directly quantifiable in economic terms. It is comparable to cost–benefit analysis in so far as it generates a ratio between benefits and costs, but is distinct in being designed intentionally to inform practical decisionmaking and to enable consideration of the sort of outcomes that have relevance in health and social care settings (Arvidson et al., 2010; Nicholls et al., 2012). SROI incorporates an economic aspect, but this is applied to serve the needs of multiple stakeholders rather than being directed principally, for example, at funders, academics or practitioners. It incorporates data that support micro-, meso- and macro levels of analysis; its application has relevance for individuals, organisations and society. Challenges will always exist with regard to demonstrating that intangible outcomes have been quantified and valued appropriately. SROI analyses must often incorporate assumptions and estimations due to their seeking to represent disparate outcomes, which might result from a combination of factors, in comparable economic terms (Arvidson et al., 2010; Gibbon and Dey, 2011). Nevertheless, this reality needs not negate the validity of that which is claimed if such elements are informed and underpinned by robust data: Rejecting absolute versions of truth, and the feasibility of absolute objectivity, is not the same as rejecting the standard of truth or the attempt to be objective. In things social and behavioural, our knowledge is always partial and intrinsically incomplete. (Needham, 1983: 32)
Conclusion Health psychology research will become more widely applicable if data are reported in a format that is relevant and meaningful for policy-makers and funders in addition to practitioners and academics (Suls and Rothman, 2004). Furthermore, encouraging all those who experience change
as a result of health interventions and innovations to provide input throughout the evaluation process is required in order to ensure that consideration can be applied appropriately to all relevant constructs (Chaudoir et al., 2013). SROI is a relatively new framework, which is still evolving, but it enables such requirements to be addressed. Expressing associated change in monetary terms allows value to be more clearly understood and provides increased clarity regarding the relative contribution of specific aspects. SROI explores all of that which results from an intervention through consideration of multiple perspectives. Some outcomes of relevance to health psychology are intangible and will always be challenging to measure and quantify. Nevertheless, this cannot be allowed to result in their value simply being disregarded. SROI remains a work in progress, but it is presented as a framework that can help health-related change to be conceptualised, understood and taken into account. SROI analyses require the application of diverse research skills that do not necessarily sit comfortably in distinct academic silos, but it is worthy of consideration as an innovative mixed methods approach that has the potential to support academic enquiry and effective endeavour. Funding This research received no specific grant from any funding agency in the public, commercial or notfor-profit sectors.
References Arvidson M, Lyon F, McKay S, et al. (2010) The ambitions and challenges of SROI. Working Paper 49, December. Birmingham: Third Sector Research Centre. Arvidson M, Lyon F, McKay S, et al. (2013) Valuing the social? The nature and controversies of measuring Social Return on Investment (SROI). Voluntary Sector Review 4(1): 3–18. Chaudoir SR, Dugan AG and Barr CHI (2013) Measuring factors affecting implementation of health innovations: A systematic review of structural, organizational, provider, patient, and innovation level measures. Implementation Science 8: 22–42.
Downloaded from hpq.sagepub.com at UNIVERSITE LAVAL on November 14, 2015
9
Leck et al. Diener E and Chan MY (2011) Happy people live longer: Subjective well-being contributes to health and longevity. Applied Psychology: Health and Well-Being 3(1): 1–43. Donovan C (2013) A Holistic Approach to Valuing Our Culture. London: DCMS. Dures E, Rumsey N, Morris M, et al. (2011) Mixed methods in health psychology: Theoretical and practical considerations of the third paradigm. Journal of Health Psychology 16(2): 332–341. Ebrahim A and Rangan KV (2010) The Limits of Nonprofit Impact: A Contingency Framework for Measuring Social Performance. Cambridge: Social Enterprise Initiative; Harvard Business School. Elkington J (1997) Cannibals with Forks: The Triple Bottom Line of 21st Century Business. Oxford: Capstone. Engel GL (1977) The need for a new medical model. Science 196(4286): 129–136. Engel GL (1980) The clinical application of the biopsychosocial model. American Journal of Psychiatry 137: 535–544. Fitzpatrick R (2000) Measurement issues in healthrelated quality of life: Challenges for health psychology. Psychology and Health 15(1): 99–108. Fujiwara D (2013) Museums and happiness: The value of participating in museums and the arts. Report, The Happy Museum Project, April. Suffolk, UK: Arts Council and Museum of East Anglian life. Available at: http://eastanglianlife. org.uk/geisha/assets/files/Museums_and_happiness_DFujiwara_April2013.pdf (accessed 10 November 2014) Fujiwara D and Campbell R (2011) Valuation Techniques for Social Cost-Benefit Analysis: Stated Preference, Revealed Preference and Subjective Well-Being Approaches – A Discussion of the Current Issues. London: HM Treasury. Gibbon J and Dey C (2011) Developments in social impact measurement in the third sector: Scaling up or dumbing down? Social and Environmental Accountability Journal 31(1): 63–72. Harlock J (2013) Impact measurement practice in the UK third sector: A review of emerging evidence. Working Paper 106, August. Birmingham: Third Sector Research Centre. Helliwell JF and Huang H (2005) How’s the job? Wellbeing and social capital in the workplace. NBER Working Paper Series, 11759. Available at: http://www.nber.org/papers/w11759.pdf?new_ window=1 (accessed 10 November 2014).
HM Government (2010) Measuring Social Value – How Five Social Enterprises Did It. London: The Stationery Office. HM Government (2011) Open Public Services. White Paper. London: The Stationery Office. HM Government (2012) Public Services (Social Value) Act. London: The Stationery Office. HM Government (2013) Open Public Services. London: The Cabinet Office. Kail A and Lumley T (2012) Theory of Change. The Beginning of Making a Difference. London: New Philanthropy Capital. Leck C (2012) Social Return on Investment (SROI) evaluation report of the Houghton Project. Report, University of Worcester, Worcester, August. Leck C, Evans N and Upton D (2014) Agriculture – Who cares? An investigation of ‘care farming’ in the UK. Journal of Rural Studies 34: 313–325. Luke B, Barraket J and Eversole R (2013) Measurement as legitimacy versus legitimacy of measures – Performance evaluation of social enterprise. In: 7th Asia Pacific interdisciplinary research in accounting conference, Kobe, Japan, 26–28 July 2010. Lyon F, Arvidson M, Etherington D, et al. (2010) Social impact measurement (SIM) experiences and future directions for third sector organisations in the East of England (Joining the dots: strand 2). Norwich:East of England Development Agency. Marks DF (1996) Health psychology in context. Journal of Health Psychology 1(1): 7–21. Martin S, Downe J, Grace C, et al. (2010) Validity, utilization and evidence-based policy: The development and impact of performance improvement regimes in local public services. Evaluation 16(1): 31–42. McPherson K, Marsh T and Brown M (2011) Tackling obesities. Future choices: modelling future trends in obesity and the impact on health. London: Government Office for Science. Available at: http://www.oxfordobesity.org/ publications/mcpherson1.pdf (accessed 10 November 2014). Milbourne L (2013) Voluntary Sector in Transition: Hard Times or New Opportunities. Bristol: Policy Press. Needham R (1983) The Tranquillity of Axiom. Los Angeles, CA: University of California Press. Nicholls J, Lawlor E, Neitzert E, et al. (2012) A Guide to Social Return on Investment (2nd edn). London: Office of the Third Sector, The Cabinet Office.
Downloaded from hpq.sagepub.com at UNIVERSITE LAVAL on November 14, 2015
10
Journal of Health Psychology
O’Carroll RE (2014) Health psychology interventions. British Journal of Health Psychology 19(2): 235–239. Owens DK, Qaseem A, Chou R, et al. (2011) Highvalue, cost-conscious health care: Concepts for clinicians to evaluate the benefits, harms, and costs of medical interventions. Annals of Internal Medicine 154(3): 174–180. Powdthavee N (2008) Putting a price tag on friends, relatives, and neighbours: Using surveys of life satisfaction to value social relationships. Journal of Socio-Economics 37(4): 1459–1480. Powdthavee N and Van den Berg B (2011) Putting different price tags on the same health condition: Re-evaluating the well-being evaluation approach. Journal of Health Economics 30(5): 1032–1043. Sarafino EP (2006) Health Psychology: Biopsychosocial Interactions (5th edn). New York: John Wiley & Sons.
Schmidt CK, Raque-Bogdan TL, Piontkowski S, et al. (2011) Putting the positive in health psychology: A content analysis of three journals. Journal of Health Psychology 16(4): 607–620. Stamuli E (2011) Health outcomes in economic evaluation: Who should value health? British Medical Bulletin 97(1): 197–210. Steed S and Nicholles N (2011) Small Slices of a Bigger Pie: Attribution in SROI. London: New Economics Foundation. Suls J and Rothman A (2004) Evolution of the biopsychosocial model: Prospects and challenges for health psychology. Health Psychology 23(2): 119–125. The National Council for Voluntary Organisations (NCVO) (2011) Participation: Trends, Facts and Figures. An NCVO Almanac. London: NCVO. Wood C and Leighton D (2010) Measuring Social Value: The Gap between Policy and Practice. London: Demos.
Downloaded from hpq.sagepub.com at UNIVERSITE LAVAL on November 14, 2015
HPQ0010.1177/1359105314557502Journal of Health PsychologyLeck et al.
Article Journal of Health Psychology 1–10 © The Author(s) 2014 Reprints and permissions: sagepub.co.uk/journalsPermissions.nav DOI: 10.1177/1359105314557502 hpq.sagepub.com
Social Return on Investment: Valuing health outcomes or promoting economic values? Chris Leck1, Dominic Upton2 and Nick Evans1
Abstract Interventions and activities that influence health are often concerned with intangible outcomes that are difficult to value despite their potential significance. Social Return on Investment is an evaluation framework that explores all aspects of change and expresses these in comparable terms. It combines qualitative narratives and quantitative measurements with a financial approach to enable outcomes that can otherwise be overlooked or undervalued to be incorporated appropriately. This article presents Social Return on Investment as an effective tool for supporting the development of a holistic appreciation of how interventions impact on the health and well-being of individuals, communities and societies.
Keywords cost-effectiveness, health psychology, outcomes, values, well-being
Introduction Interventions and activities that have relevance to health psychology can be multifactorial and concerned with intangible outcomes that are challenging to identify and measure, let alone to conceptualise meaningfully (Fitzpatrick, 2000; O’Carroll, 2014; Powdthavee and Van den Berg, 2011; Stamuli, 2011). This can result in elements of associated value (negative and positive) being overlooked, misunderstood or inappropriately assigned, despite their perhaps having significance in relation to individual and collective lives (Nicholls et al., 2012; Owens et al., 2011). As Owens et al. (2011) observed, ‘judgments about value are fundamental to decision making in most arenas but often have been missing in health care settings’ (p. 174). Health psychology needs to be able to identify and articulate the changes in behaviour and
circumstances that occur as a result of interventions and activities (Fitzpatrick, 2000; O’Carroll, 2014). Numerous studies have indicated over at least the last four decades that health and wellbeing are influenced by a combination of social, environmental and economic factors (Suls and Rothman, 2004). Therefore, research tools are required that can incorporate consideration of that which has relevance to all such discourses rather than focusing solely on those aspects of change that are pre-defined or immediately 1University 2University
of Worcester, UK of Canberra, Australia
Corresponding author: Chris Leck, University of Worcester, Bredon 65, Henwick Grove, Worcester WR2 6AJ, UK. Email: [email protected]
Downloaded from hpq.sagepub.com at UNIVERSITE LAVAL on November 14, 2015
2
Journal of Health Psychology
quantifiable (Marks, 1996; Schmidt et al., 2011). ‘In short, health psychologists need to design research that embraces rather than shies away from the complexity of the phenomena of interest’ (Suls and Rothman, 2004: 123). Social Return on Investment (SROI) is an outcome-focused framework that seeks to enable such research and is being applied increasingly for evaluation purposes (Arvidson et al., 2010; Ebrahim and Rangan, 2010). It combines qualitative narratives and quantitative measurements with a financial approach to enable associated change to be identified, explored, conceptualised and valued in comparable terms (Arvidson et al., 2013; Nicholls et al., 2012). SROI is intended to serve the needs of those who access, provide, fund and evaluate services and interventions (Arvidson et al., 2010; Harlock, 2013; HM Government, 2010; Martin et al., 2010; Nicholls et al., 2012; Wood and Leighton, 2010). SROI analyses seek to take account of all social, environmental and economic outcomes – the ‘triple bottom line’ (Elkington, 1997) – and articulate their associated value in monetary terms (Arvidson et al., 2010; Harlock, 2013; Nicholls et al., 2012). A return on investment ratio is calculated, and this is the element that is often highlighted, but SROI is intended to be about more than generating an economic interpretation of value (Arvidson et al., 2013; Nicholls et al., 2012). Quantitative data and qualitative explorations of real-world contexts and experiences are combined to provide an enhanced understanding of reality (Dures et al., 2011; Nicholls et al., 2012). SROI supports a biopsychosocial approach to health as it allows biological and psychological factors to be considered in conjunction with the relevance of the social and physical world that is inhabited (Engel, 1977, 1980; Sarafino, 2006). A large volume of grey literature detailing the application of SROI in the health and social care sector is now available, but it is an impact measurement methodology that has not previously received scrutiny with regard to its relevance and applicability for the purposes of health psychology. This reality needs to be
addressed in recognition of the fact that, ‘as the use of economic evaluation studies continues to grow, it is important that patients, clinicians and decision makers, as well as the general public, understand the key value judgements which lie behind these assessments’ (Stamuli, 2011: 208). This article supports the development of such an understanding by describing the principles and process that underpin SROI. It outlines techniques that are applied to measure and quantify the sort of intangible or ‘soft’ outcomes (such as improved mood or enhanced social inclusion) that are difficult to value but contribute towards health and well-being. The strengths and challenges associated with the methodology are discussed to assess its suitability as a framework to support the requirements of health psychology. The article references a case study to help illustrate how SROI has been applied practically to support the evaluation of a healthrelated intervention (Leck, 2012). The study under consideration relates to a UK care farm and has been assured by the SROI Network. Many definitions of care farming are available, reflecting specific national policy contexts, but it concerns vulnerable people being enabled to engage with farming activities to promote their health and well-being (Leck et al., 2014). As the descriptor for the activity suggests, the provision of care is an integral element, but this is not unidirectional. The intent is to enable participants to care for themselves and one another in addition to the livestock, crops and wider ‘natural’ environment (Leck et al., 2014). Participants are encouraged to learn new skills and engage in productive work within an intentionally inclusive and therapeutic place.
Applying SROI to understand impact The approach to SROI that informs this article is outlined in ‘A guide to social return on investment’ (Nicholls et al., 2012). This publication was originally funded by the UK Government Cabinet Office in 2009, but has since been updated. The following principles and stages
Downloaded from hpq.sagepub.com at UNIVERSITE LAVAL on November 14, 2015
3
Leck et al. are highlighted in both editions as fundamental to the overall process: SROI principles: 1. 2. 3. 4. 5. 6. 7.
Involve material stakeholders Understand what changes Value what matters Include only what is material Avoid over-claiming Be transparent Verify the result
SROI process: 1. 2. 3. 4. 5. 6.
Establish scope and key stakeholders Map outcomes Evidence and value outcomes Establish impact Calculate the SROI Report, use and embed
Stakeholder identification and involvement are incorporated as the first stage and principle of SROI. The term ‘stakeholder’ refers to all those who are identified as potentially experiencing change because of the activities under consideration. These might, for example, include individuals who are directly involved (such as service users and volunteers), members of personal support networks (such as family and carers), organisations (such as hospitals and social care providers) and wider society (as a result, for example, of reduced benefit payments or increased social cohesion). The scope of an SROI is clarified at the outset and identifies the activities to be included, the period of time to which it relates and the type of analysis to be applied (evaluation or forecast). All potentially relevant stakeholder groups are identified and their materiality for the purposes of the SROI is assessed according to the form and extent of anticipated change (Nicholls et al., 2012). However, it is always possible that significant stakeholder groups will be overlooked or undervalued during the initial stages of an evaluation. SROI requires that relevance is continuously reassessed to ensure that
all those who experience change can be taken into consideration (Arvidson et al., 2010; Nicholls et al., 2012). The development of an impact map is a central feature of SROI. This articulates a theory of change and encompasses the data that inform the calculation of the return on investment ratio (Nicholls et al., 2012). Theory of change is a research approach that sits well within the SROI framework because it links activities to outcomes and supports the development of a qualitative awareness of relevant relationships from the outset (Kail and Lumley, 2012). Inputs and outputs that relate to all material stakeholder groups are initially identified and recorded on the impact map (Nicholls et al., 2012). While the former encompass that which is invested (e.g. salaries paid), the latter incorporate information about that which is provided (e.g. number of sessions delivered). This is generally a fairly straightforward part of the process, with informing quantitative data being provided by practitioners and service providers. The identification of outcomes, and the selection of indicators and proxies that conceptualise these, is more complex. Processes and outcomes are often explored initially through qualitative consultations with stakeholders, but quantitative data pertaining to appropriate indicators evidence the extent to which they apply and ascertain associated value (Nicholls et al., 2012). If, for example, an outcome relates to increased self-confidence, then an appropriate indicator might concern someone engaging in new social activities. An over-reliance on selfreported (subjective) measures can be avoided by also obtaining supporting quantitative data and evidence from more objective third-party sources. Outcomes are valued through the application of justifiable proxies that have monetary value (Arvidson et al., 2010; Nicholls et al., 2012). These can be identified from a variety of external sources (such as databases, academic analyses and other SROI studies), but it is imperative that they are recognised by those to whom they apply as being appropriate and realistic economic valuations of the change that they experience.
Downloaded from hpq.sagepub.com at UNIVERSITE LAVAL on November 14, 2015
4
Journal of Health Psychology
The proxies that are applied can only ever be surrogates, and some variation will always exist between individuals, but this element of the process is intended to enable outcomes that might otherwise be overlooked to be afforded appropriate recognition (Nicholls et al., 2012). Crucially, it can allow ‘intangible’ outcomes (alternatively described as ‘soft’ or ‘indirect’), which do not have direct economic value, but can be critical to health and well-being, to be explored and incorporated in the analysis (Arvidson et al., 2010, 2013; Diener and Chan, 2011; Wood and Leighton, 2010). Outcomes concern the changes that occur during the period of time under consideration, but impact relates to the contribution that the intervention itself makes to this change (Steed and Nicholles, 2011). This distinction is easily overlooked when interventions are being evaluated and can result in a distorted picture being presented. SROI incorporates procedures intended specifically to allow external influences to be taken into account and excluded (Arvidson et al., 2013; Nicholls et al., 2012). These identify that which would have happened regardless of the intervention taking place (deadweight), has occurred as a result of other input (attribution) or that has moved rather than being removed (displacement). Attribution can be particularly relevant in relation to complex health and social care issues that often involve input from multiple agencies (Steed and Nicholles, 2011). The figures provided can only ever be estimates, but they are, once again, underpinned by evidence provided by the stakeholders concerned. Additionally, SROI acknowledges that the effects of some outcomes last longer than others. While some are dependent on the activity continuing, others might continue to exert influence, albeit of a diminishing form, into the future. In such instances, and assuming sufficient evidence is available to adequately justify such a claim, duration can be extended beyond the time period under consideration and the value applied to the related change increased accordingly. All outcome values (positive and negative) that are attributable to the activity under
consideration are added together on the impact map and compared to the original investment to provide the SROI ratio (Nicholls et al., 2012). Sensitivity analysis is subsequently applied to test the overall effect on the final return ratio of particular outcomes/proxies and associated estimates/assumptions. Findings are then shared with stakeholders to ensure that they perceive that which is reported as accurately reflecting their own reality. Consideration will now be given to the practical application of SROI. Associated strengths and challenges are explored in order to assess the extent to which it is a framework that has applicability in social and health-care settings. Specific elements of the previously referenced care farm SROI are incorporated to illustrate that which is described.
Material stakeholders It is a central, distinctive and defining feature of SROI that consideration is applied to all material stakeholders (Arvidson et al., 2010; Nicholls et al., 2012). This helps ensure that all related outcomes are valued in the analysis rather than solely those that apply directly to the primary intended beneficiary. For example, data collected for the care farm SROI identified significant associated value for the National Health Service (NHS) and the carers and family members of the service users. This latter group indicated that relationships had improved, and that they had benefited from having time to relax and recuperate, comfortable in the knowledge that the person they usually took responsibility for supporting was happy, safe and actively engaged elsewhere. Most care farm service users had previously required in-patient hospital treatment in connection with their condition, but only one individual had needed to be readmitted since attending the farm (Leck, 2012). Additionally, the care farm was found to enable physical exercise and participating in food production (from field to fork) was presented as encouraging people to adopt a healthy diet and lifestyle. Given that body mass index (BMI)–related illnesses have
Downloaded from hpq.sagepub.com at UNIVERSITE LAVAL on November 14, 2015
5
Leck et al. been calculated to cost the NHS 17.4 billion pounds sterling per annum (McPherson et al., 2011), associated savings were judged to be significant and worthy of inclusion. Various other changes that will result similarly in cost savings at a national level were also identified in the care farm case study. These included reduced benefit payments, drug/alcohol use, social/educational support needs and public disorder/crime. However, they were not found to be sufficiently widespread to justify inclusion on the impact map. This will have resulted in the final SROI ratio being lower than might otherwise have been the case, and the significance of such change for the individuals concerned still needed to be highlighted, but it is not practical to incorporate outcomes on a case-by-case basis. Only 61 per cent of the total value associated with the care farm was found ultimately to relate to the service users themselves. Another 18 per cent concerned their carers/relatives and 17 per cent took the form of savings to the NHS resulting from reduced use of services (Leck, 2012). This latter outcome is unlikely to translate into direct and immediate financial savings, but value is derived from resources being freed for application elsewhere. It is a particular strength of SROI, and something that is often absent in relation to project evaluations, that it can provide such an understanding of wherein value actually lies.
Understanding outcomes SROI is a framework that can support health researchers in describing and quantifying the wider value that interventions can provide for individuals, communities and society. A holistic approach is adopted that allows a meaningful understanding of what is really happening to be provided rather than exploring solely that which is pre-defined or anticipated (Nicholls et al., 2012). SROI embraces consideration of outcomes that are not immediately quantifiable rather than focusing exclusively on those that are more easily measured.
Intentionally incorporated flexibility helps ensure that it is not only intended outcomes that are identified but also those that are unintended and can be overlooked if performance is evaluated solely according to preordained objectives and measures (Nicholls et al., 2012). All stakeholders are directly involved throughout the process and unanticipated or unintended outcomes can be explored and incorporated effectively in conjunction with that which has been pre-defined. Multiple outcomes that influence health and well-being will often be cited or observed during an evaluation process and this has the potential to result in excessive claims being applied if all are valued independently. With regard to the care farm SROI, relevant outcomes were found to include increased levels of relaxation, happiness, satisfaction, confidence, social engagement and feelings of self-worth. However, it would have been neither realistic nor appropriate to value each of these outcomes separately. This would have increased the return on investment ratio, but it would also have resulted effectively in the same changes being counted on more than one occasion. SROI manages this risk by incorporating such related changes within chains of events rather than as outcomes to be valued individually (Nicholls et al., 2012). The following chains were developed in conjunction with the care farm service users to accommodate the key outcomes that were found to apply among this group of stakeholders: Service users got structure to their day → learnt new skills → did something productive in the workplace (meaningful activity) → became more confident → received job satisfaction. Service users enjoyed being in a restorative natural environment → became more relaxed → felt happier → interacted with others → made friends → became part of a supportive and reciprocal social network. Service users were active on the farm → adopted a healthier lifestyle and diet → physical health improved.
Downloaded from hpq.sagepub.com at UNIVERSITE LAVAL on November 14, 2015
6
Journal of Health Psychology
Financial proxies were only assigned on the care farm impact map to the most advanced stage of these chains of events (Nicholls et al., 2012). This process allowed all relevant outcomes to be evidenced and incorporated into the wider analysis while also reducing the risk of an inaccurate picture being provided.
Assigning proxy values The assigning of monetary values to outcomes is an element of SROI that is both fundamental and contentious, and no consensus exists regarding the most appropriate techniques (Donovan, 2013; Fujiwara and Campbell, 2011; Powdthavee and Van den Berg, 2011). Contingent valuation methods have been applied fairly widely, with SROI analyses having utilised both ‘revealed preference’ and ‘stated preference’ techniques (Fujiwara and Campbell, 2011; HM Government, 2010). Revealed preference techniques observe how people behave in related markets (hedonic pricing) or calculate costs associated with accessing services, but these can be challenging to apply in relation to the sort of intangible outcomes that have relevance to health and well-being (Fujiwara and Campbell, 2011; Powdthavee and Van den Berg, 2011). Some stated preference techniques involve value being assigned via the concepts of ‘willingness to pay’ for an outcome to happen or ‘willingness to accept’ in terms of the compensation required to replace the outcome (Fujiwara and Campbell, 2011; Powdthavee and Van den Berg, 2011). Choice modelling is another stated preference strategy, that allows respondents to be provided with various alternatives that they can then rank (Donovan, 2013). Associated exercises can incorporate the outcome concerned alongside other changes that have more apparent financial value. These might, for example, include things such as a new pair of shoes, a new car or a holiday. The ordering of responses can help conceptualise the relative value attributed to the outcome concerned by the person to whom it applies.
Another method for valuing intangible outcomes that influence health and well-being is to apply statistical analyses to large national data sets (Fujiwara, 2013; Powdthavee and Van den Berg, 2011). Measured effects, relating for example to happiness or quality of life, are on this occasion compared with the impact associated with additional income through the application of regression analysis (Arvidson et al., 2013; Fujiwara and Campbell, 2011). This technique has been presented as more accurately reflecting personal realities as opposed to that which is imagined to apply (Fujiwara and Campbell, 2011). However, valuations for the same health condition can vary fairly substantially according to the well-being measure that is applied (Powdthavee and Van den Berg, 2011). With regard to the care farm case study, paid employment in a competitive marketplace was not currently a realistic option for many participants. Nevertheless, and as indicated previously, those concerned obtained job satisfaction from their active participation on the farm. A 1-point rise in job satisfaction (on a 10-point scale) has been calculated as equating to 36 per cent of income (Helliwell and Huang, 2005), and this figure was on this occasion applied to the minimum UK wage to obtain an appropriate proxy value. Given the fact that proxy values can be obtained via a range of methods, it is essential that those that are applied are recognised as appropriate by those to whom they apply. Becoming part of a genuine and effective social network was also valued for the purposes of the care farm SROI according to data obtained from the application of statistical analysis to a large national data set, but the stakeholders concerned did not on this occasion feel that associated figures represented their reality. A shadow pricing method that was applied to the British Household Panel Survey had presented increased social involvement as worth the equivalent of up to £85,000 per annum, with a more modest figure of £15,500 being suggested to reflect the value of seeing friends or relatives on most days of the week rather than one or two (Powdthavee, 2008). However, even this lower
Downloaded from hpq.sagepub.com at UNIVERSITE LAVAL on November 14, 2015
7
Leck et al. figure was considered excessive by care farm participants. Discussions with those concerned identified 25 per cent of this latter figure as more closely equating to their reality. Conceptualising less tangible outcomes in economic terms will always incorporate challenges and be dependent to some extent on personally informed value judgments, but this aspect has been suggested additionally to distract attention from the evidence that underpins the analysis and increase the likelihood of research being used inappropriately (Arvidson et al., 2010; Gibbon and Dey, 2011; Harlock, 2013). Consideration is now given to the validity of such claims.
Measuring value The SROI Guide emphasises that the ratio of return on investment should not be the sole reason for engaging with SROI, considered in isolation or compared between organisations (Arvidson et al., 2010, 2013; Lyon et al., 2010; Nicholls et al., 2012). However, current austerity measures and competitive realities in the United Kingdom and elsewhere are contributing towards health and social care policy-makers and funders focusing increasingly on the maximisation of the return that their investment provides (Arvidson et al., 2013). This is suggested to increase the risk of undue emphasis being placed on the ratio, comparisons being drawn and attempts being made to inflate or hide relevant aspects (Arvidson et al., 2013; Gibbon and Dey, 2011; Milbourne, 2013). Such realities are undeniable, and debates regarding the application of financial indicators to outcomes that do not have a direct monetary value will continue, but the relevance of market-led economic processes in the health sector has to be acknowledged (Gibbon and Dey, 2011). ‘The economic evaluation of healthcare interventions has become an integral part of health policy and decision-making for the allocation of the limited healthcare resources’ (Stamuli, 2011: 197). Incorporated proxies cannot be presented as, or claimed to be, precisely
applicable in all instances – the extent of associated change and the degree to which it is valued will always vary between individuals – but their inclusion helps ensure that the value of that which is not immediately quantifiable, but is nevertheless relevant, can be afforded appropriate recognition. It is understandable that particular attention is given to the ratio of return because it is a distinctive feature of SROI, allows total associated value to be more easily conceptualised and is a useful peg on which to hang an exploration of impact. However, this number is not intended to be considered in isolation (Nicholls et al., 2012). The reductive and quantitative element provided by the ratio should be applied and interpreted in conjunction with the summative assessment that is more qualitatively informed. The intent is not simply to express associated value numerically but rather to provide a holistic understanding of that which takes place, associated processes and relative import. The presentation or perception of SROI as a market-based evaluation tool concerned solely with monetary value will not sit comfortably with those who engage with health and social interventions that are not for profit or have philanthropic intent (Arvidson et al., 2013). Nevertheless, it is naive and unrealistic to imagine that economic realities do not apply to health and social care (Owens et al., 2011). The UK government is a significant funder in the sector, promotes competitive tendering and requires impact to be demonstrable (HM Government, 2011, 2012, 2013; The National Council for Voluntary Organisations, 2011). Equating SROI with the marketisation of the subject of the evaluation is failing to understand its fundamental purpose. It is intended to be about value rather than money and to measure value rather than merely value measures (Luke et al., 2013; Nicholls et al., 2012). SROI seeks to enable all that which takes place as a result of an activity to be taken into consideration rather than solely that which has direct applicability in the marketplace. The purpose is to articulate the current value while also providing an understanding of the sources of that value.
Downloaded from hpq.sagepub.com at UNIVERSITE LAVAL on November 14, 2015
8
Journal of Health Psychology
In many ways, the elements of SROI that present the greatest challenges are those that can be perceived also as providing its strengths. It is a framework that seeks to financially quantify all elements of value rather than solely those that are directly quantifiable in economic terms. It is comparable to cost–benefit analysis in so far as it generates a ratio between benefits and costs, but is distinct in being designed intentionally to inform practical decisionmaking and to enable consideration of the sort of outcomes that have relevance in health and social care settings (Arvidson et al., 2010; Nicholls et al., 2012). SROI incorporates an economic aspect, but this is applied to serve the needs of multiple stakeholders rather than being directed principally, for example, at funders, academics or practitioners. It incorporates data that support micro-, meso- and macro levels of analysis; its application has relevance for individuals, organisations and society. Challenges will always exist with regard to demonstrating that intangible outcomes have been quantified and valued appropriately. SROI analyses must often incorporate assumptions and estimations due to their seeking to represent disparate outcomes, which might result from a combination of factors, in comparable economic terms (Arvidson et al., 2010; Gibbon and Dey, 2011). Nevertheless, this reality needs not negate the validity of that which is claimed if such elements are informed and underpinned by robust data: Rejecting absolute versions of truth, and the feasibility of absolute objectivity, is not the same as rejecting the standard of truth or the attempt to be objective. In things social and behavioural, our knowledge is always partial and intrinsically incomplete. (Needham, 1983: 32)
Conclusion Health psychology research will become more widely applicable if data are reported in a format that is relevant and meaningful for policy-makers and funders in addition to practitioners and academics (Suls and Rothman, 2004). Furthermore, encouraging all those who experience change
as a result of health interventions and innovations to provide input throughout the evaluation process is required in order to ensure that consideration can be applied appropriately to all relevant constructs (Chaudoir et al., 2013). SROI is a relatively new framework, which is still evolving, but it enables such requirements to be addressed. Expressing associated change in monetary terms allows value to be more clearly understood and provides increased clarity regarding the relative contribution of specific aspects. SROI explores all of that which results from an intervention through consideration of multiple perspectives. Some outcomes of relevance to health psychology are intangible and will always be challenging to measure and quantify. Nevertheless, this cannot be allowed to result in their value simply being disregarded. SROI remains a work in progress, but it is presented as a framework that can help health-related change to be conceptualised, understood and taken into account. SROI analyses require the application of diverse research skills that do not necessarily sit comfortably in distinct academic silos, but it is worthy of consideration as an innovative mixed methods approach that has the potential to support academic enquiry and effective endeavour. Funding This research received no specific grant from any funding agency in the public, commercial or notfor-profit sectors.
References Arvidson M, Lyon F, McKay S, et al. (2010) The ambitions and challenges of SROI. Working Paper 49, December. Birmingham: Third Sector Research Centre. Arvidson M, Lyon F, McKay S, et al. (2013) Valuing the social? The nature and controversies of measuring Social Return on Investment (SROI). Voluntary Sector Review 4(1): 3–18. Chaudoir SR, Dugan AG and Barr CHI (2013) Measuring factors affecting implementation of health innovations: A systematic review of structural, organizational, provider, patient, and innovation level measures. Implementation Science 8: 22–42.
Downloaded from hpq.sagepub.com at UNIVERSITE LAVAL on November 14, 2015
9
Leck et al. Diener E and Chan MY (2011) Happy people live longer: Subjective well-being contributes to health and longevity. Applied Psychology: Health and Well-Being 3(1): 1–43. Donovan C (2013) A Holistic Approach to Valuing Our Culture. London: DCMS. Dures E, Rumsey N, Morris M, et al. (2011) Mixed methods in health psychology: Theoretical and practical considerations of the third paradigm. Journal of Health Psychology 16(2): 332–341. Ebrahim A and Rangan KV (2010) The Limits of Nonprofit Impact: A Contingency Framework for Measuring Social Performance. Cambridge: Social Enterprise Initiative; Harvard Business School. Elkington J (1997) Cannibals with Forks: The Triple Bottom Line of 21st Century Business. Oxford: Capstone. Engel GL (1977) The need for a new medical model. Science 196(4286): 129–136. Engel GL (1980) The clinical application of the biopsychosocial model. American Journal of Psychiatry 137: 535–544. Fitzpatrick R (2000) Measurement issues in healthrelated quality of life: Challenges for health psychology. Psychology and Health 15(1): 99–108. Fujiwara D (2013) Museums and happiness: The value of participating in museums and the arts. Report, The Happy Museum Project, April. Suffolk, UK: Arts Council and Museum of East Anglian life. Available at: http://eastanglianlife. org.uk/geisha/assets/files/Museums_and_happiness_DFujiwara_April2013.pdf (accessed 10 November 2014) Fujiwara D and Campbell R (2011) Valuation Techniques for Social Cost-Benefit Analysis: Stated Preference, Revealed Preference and Subjective Well-Being Approaches – A Discussion of the Current Issues. London: HM Treasury. Gibbon J and Dey C (2011) Developments in social impact measurement in the third sector: Scaling up or dumbing down? Social and Environmental Accountability Journal 31(1): 63–72. Harlock J (2013) Impact measurement practice in the UK third sector: A review of emerging evidence. Working Paper 106, August. Birmingham: Third Sector Research Centre. Helliwell JF and Huang H (2005) How’s the job? Wellbeing and social capital in the workplace. NBER Working Paper Series, 11759. Available at: http://www.nber.org/papers/w11759.pdf?new_ window=1 (accessed 10 November 2014).
HM Government (2010) Measuring Social Value – How Five Social Enterprises Did It. London: The Stationery Office. HM Government (2011) Open Public Services. White Paper. London: The Stationery Office. HM Government (2012) Public Services (Social Value) Act. London: The Stationery Office. HM Government (2013) Open Public Services. London: The Cabinet Office. Kail A and Lumley T (2012) Theory of Change. The Beginning of Making a Difference. London: New Philanthropy Capital. Leck C (2012) Social Return on Investment (SROI) evaluation report of the Houghton Project. Report, University of Worcester, Worcester, August. Leck C, Evans N and Upton D (2014) Agriculture – Who cares? An investigation of ‘care farming’ in the UK. Journal of Rural Studies 34: 313–325. Luke B, Barraket J and Eversole R (2013) Measurement as legitimacy versus legitimacy of measures – Performance evaluation of social enterprise. In: 7th Asia Pacific interdisciplinary research in accounting conference, Kobe, Japan, 26–28 July 2010. Lyon F, Arvidson M, Etherington D, et al. (2010) Social impact measurement (SIM) experiences and future directions for third sector organisations in the East of England (Joining the dots: strand 2). Norwich:East of England Development Agency. Marks DF (1996) Health psychology in context. Journal of Health Psychology 1(1): 7–21. Martin S, Downe J, Grace C, et al. (2010) Validity, utilization and evidence-based policy: The development and impact of performance improvement regimes in local public services. Evaluation 16(1): 31–42. McPherson K, Marsh T and Brown M (2011) Tackling obesities. Future choices: modelling future trends in obesity and the impact on health. London: Government Office for Science. Available at: http://www.oxfordobesity.org/ publications/mcpherson1.pdf (accessed 10 November 2014). Milbourne L (2013) Voluntary Sector in Transition: Hard Times or New Opportunities. Bristol: Policy Press. Needham R (1983) The Tranquillity of Axiom. Los Angeles, CA: University of California Press. Nicholls J, Lawlor E, Neitzert E, et al. (2012) A Guide to Social Return on Investment (2nd edn). London: Office of the Third Sector, The Cabinet Office.
Downloaded from hpq.sagepub.com at UNIVERSITE LAVAL on November 14, 2015
10
Journal of Health Psychology
O’Carroll RE (2014) Health psychology interventions. British Journal of Health Psychology 19(2): 235–239. Owens DK, Qaseem A, Chou R, et al. (2011) Highvalue, cost-conscious health care: Concepts for clinicians to evaluate the benefits, harms, and costs of medical interventions. Annals of Internal Medicine 154(3): 174–180. Powdthavee N (2008) Putting a price tag on friends, relatives, and neighbours: Using surveys of life satisfaction to value social relationships. Journal of Socio-Economics 37(4): 1459–1480. Powdthavee N and Van den Berg B (2011) Putting different price tags on the same health condition: Re-evaluating the well-being evaluation approach. Journal of Health Economics 30(5): 1032–1043. Sarafino EP (2006) Health Psychology: Biopsychosocial Interactions (5th edn). New York: John Wiley & Sons.
Schmidt CK, Raque-Bogdan TL, Piontkowski S, et al. (2011) Putting the positive in health psychology: A content analysis of three journals. Journal of Health Psychology 16(4): 607–620. Stamuli E (2011) Health outcomes in economic evaluation: Who should value health? British Medical Bulletin 97(1): 197–210. Steed S and Nicholles N (2011) Small Slices of a Bigger Pie: Attribution in SROI. London: New Economics Foundation. Suls J and Rothman A (2004) Evolution of the biopsychosocial model: Prospects and challenges for health psychology. Health Psychology 23(2): 119–125. The National Council for Voluntary Organisations (NCVO) (2011) Participation: Trends, Facts and Figures. An NCVO Almanac. London: NCVO. Wood C and Leighton D (2010) Measuring Social Value: The Gap between Policy and Practice. London: Demos.
Downloaded from hpq.sagepub.com at UNIVERSITE LAVAL on November 14, 2015
