Social Science & Medicine 114 (2014) 103e112

Contents lists available at ScienceDirect

Social Science & Medicine journal homepage: www.elsevier.com/locate/socscimed

Evaluating return on investment in a school based health promotion and prevention program: The investment multiplier for the Stephanie Alexander Kitchen Garden National Program Simon Eckermann a, *, James Dawber a, Heather Yeatman b, Karen Quinsey a, Darcy Morris a a b

Australian Health Services Research Institute, University of Wollongong, Wollongong 2522, NSW, Australia School of Health and Society, Faculty of Social Sciences, University of Wollongong, Wollongong, NSW, Australia

a r t i c l e i n f o

a b s t r a c t

Article history: Received 27 August 2013 Received in revised form 20 May 2014 Accepted 30 May 2014 Available online 2 June 2014

Successful health promotion and disease prevention strategies in complex community settings such as primary schools rely on acceptance and ownership across community networks. Assessing multiplier impacts from investment on related community activity over time are suggested as key alongside evidence of program health effects on targeted groups of individuals in gauging community network engagement and ownership, dynamic impacts, and program long term success and return on investment. An Australian primary school based health promotion and prevention strategy, the Stephanie Alexander Kitchen Garden National Program (SAKGNP), which has been providing garden and kitchen classes for year 3e6 students since 2008, was evaluated between 2011 and 2012. Returns on Australian Federal Government investment for school infrastructure grants up to $60,000 are assessed up to and beyond a two year mutual obligation period with: (i) Impacts on student lifestyle behaviours, food choices and eating habits surveyed across students (n ¼ 491 versus 260) and parents (n ¼ 300 versus 234) in 28 SAKGNP and 14 matched schools, controlling for school and parent level confounders and triangulated with SAKGNP pre-post analysis; (ii) Multiplier impacts of investment on related school and wider community activity up to two years; and (iii) Evidence of continuation and program evolution in schools observed beyond two years. SAKGNP schools showed improved student food choices (p ¼ 0.024) and kitchen lifestyle behaviour (p ¼ 0.019) domains compared to controls and in pre-post analysis where 20.0% (58/290) reported eating fruit and vegetables more often and 18.6% (54/290) preparing food at home more often. No significant differences were found in case control analysis for eating habits or garden lifestyle behaviour domains, although 32.3% of children helped more in the garden (91/278) and 15.6% (45/289) ate meals together more often in pre-post analysis. The multiplier impact on total community activity up to two years was 5.07 ($226,737/$44,758); 1.60 attributable to school, and 2.47 to wider community, activity. All 8 schools observed beyond two years continued garden and kitchen classes, with an average 17% scaling up and one school fully integrating staff into the curriculum. In conclusion evidence supports the SAKGNP to be a successful health promotion program with high community network impacts and return on investment in practice. © 2014 Published by Elsevier Ltd.

Keywords: Australia Health promotion and prevention Lifestyle and eating behaviours Complex community-based interventions Investment multiplier Health economics Stephanie Alexander Kitchen Garden National Program

* Corresponding author. E-mail address: [email protected] (S. Eckermann). http://dx.doi.org/10.1016/j.socscimed.2014.05.056 0277-9536/© 2014 Published by Elsevier Ltd.

104

S. Eckermann et al. / Social Science & Medicine 114 (2014) 103e112

1. Introduction 1.1. Evaluating success and return on investment for health promotion and prevention programs in complex community system settings such as primary schools Health promotion and prevention strategies in community settings are in general most effective where they engage with social networks and build social capital to enable community ownership and embedding of strategies (Hawe and Shiell, 2000; Moore et al., 2006). This is particularly the case for health promotion and prevention activities at primary schools, as social institutions within a community setting that represent a complex system with direct and indirect community relationships and social networks (Hawe and Ghali, 2008; Shiell et al., 2008). In such complex systems engagement with and ownership of programs by networks and dynamic impacts between programs and networks are key to the success of health promotion and prevention strategies (Shiell and Hawe, 1995). There is an increasing body of evidence that independent of evidence of potential impacts at an individual level, health promotion and prevention programs that do not engage with community networks and ownership, fail over time to have an impact on population health, particularly beyond program obligation and evaluation periods (Hawe et al., 2009). Without community involvement and ownership, impacts are short-lived (Schensul, 2005) and fail to be effective in impacting on disadvantaged populations targeted (Hill et al., 2005). While the Framingham study (Dawber et al., 1957; Truett et al., 1967) showed major causes of death are preventable, individual patient rather than community level programs have had marginal if any impact (Zaza et al., 2005). Success of health promotion programs in community settings such as schools generally requires engaging with complex systems of networks in those communities (Shiell et al., 2008). Where behaviour models have been successful at population levels, such as telephone messaging in improving physical activity and diet (Eakin et al., 2007), they have generally coincided with approaches which also actively engage with and enable dynamic impacts over time across networks in communities. Hence, conventional cost effectiveness analysis models informed by evaluation of effects on individual patients without consideration of community network impacts struggle to gauge the expected long term effects of health promotion and prevention programs in such community settings (Shiell and Hawe, 1995). Conventional assessment of within study costs and effects typically do not enable assessment of whether programs will continue beyond evaluation periods, are expected to be successful or provide an adequate return on investment, with long term effects usually postulated around sensitivity and scenario analysis. However, multiplier effects from program investment flowing across networks into community activities in such settings have been suggested to provide a robust quantitative indicator of community ownership, engagement with, and building of, social networks and capital, and sustainability of programs over time (Hawe et al., 2009; Shiell et al., 2008). Multiplier assessment over time provides key evidence to assess network engagement, ownership and dynamic impacts. Triangulated with qualitative evidence of impacts of context this enables informed assessment of whether typically short term program effects on individuals during evaluation periods can be expected to translate into sustainable programs with long term outcomes across communities. Such triangulated and combined assessment of individual and community impacts is key in health promotion, as Periago highlights in the Guide to economic evaluation in health promotion: “To make a true economic assessment of health

promotion options, one must be forward thinking and consider many different avenues to arrive at a given result” (de Salazar et al., 2007: p. 1). Estimating multiplier effects over time also provides a foundation to consider more complex forms of network analysis, such as mapping activity impacts to the role of gatekeepers or champions, feedback loops between context and intervention, or the extensiveness and intensiveness (Yin, 1979) of networks over time (Hawe et al., 2009), which we return to consider in discussion. 1.2. The Stephanie Alexander Kitchen Garden National Program (SAKGNP) in primary schools The Stephanie Alexander Kitchen Garden National Program (SAKGNP) is a school based program designed to promote pleasurable food education in Australian primary schools (Stephanie Alexander Kitchen Garden Foundation, 2011). Since 2008 the Australian Federal Government has funded competitive capital grants of up to $60,000 for Australian primary schools to build garden and kitchen facilities, with a mutual obligation for funded schools to undertake Stephanie Alexander Kitchen Garden (SAKG) classes for primary school students aged 8e12 years (across grades 3e6) for two years. To be eligible for SAKGNP capital funding, schools in a grant application process needed to demonstrate a whole of school commitment to: hiring garden and kitchen specialists to support weekly lessons for all Years 3e6 students (45 min each school week in a vegetable garden and 90 min in a kitchen classroom); linking lessons to the official curriculum; support and engagement in the Program, including involvement of community volunteers; and a minimum two year commitment to running SAKGNP classes. The SAKGNP represents the extension of a pilot SAKG pilot program in Victoria which started in 2001. The Victorian SAKG pilot program had also competitively funded primary schools for capital expenditure in setting up gardens and kitchens, but with additional recurrent expenditure associated with running classes over the first 12 months (Block et al., 2012; Block and Johnson, 2009; Gibbs et al., 2013a, 2013b). A national evaluation of the SAKGNP was commissioned in late 2011 to determine the health-related impacts and program-related outcomes including return on investment to the Australian government (Yeatman et al., 2013, in press). This paper considers the methods, analysis and findings for evaluating the return on investment component of the SAKGNP from a societal perspective, allowing for impacts on students, parents, staff, schools and wider communities. 2. Methods Ethical approval for the evaluation was obtained from the University of Wollongong Human Research Ethics Committee and all relevant jurisdictional education departments. Short term impacts of the SAKGNP on student attitudes, behaviour and lifestyle are evaluated with triangulation of comparative and pre-post surveys of students and parents [INSERT LINK TO ONLINE FILE A] for four SAKGNP domains of interest proposed by the Australian Department of Health and Ageing: (i) Garden lifestyle behaviours consider the level of enjoyment, confidence and ability of students in gardening; (ii) Kitchen lifestyle behaviours consider the level of enjoyment, confidence and ability of student in cooking and other kitchen activities; (iii) Eating habits and behaviours consider the quantity of fruit and vegetables consumed and mealtime behaviours; (iv) Food choices consider willingness to try new foods and diversity of foods chosen to eat.

S. Eckermann et al. / Social Science & Medicine 114 (2014) 103e112

Network impacts of the SAKGNP are evaluated by estimating the multiplier effects of initial investment on direct school and wider community health promotion related activity up to and beyond the two year SAKGNP commitment. This enables evaluation of the extent of continuation and embedding of the SAKGNP over time, which is triangulation with qualitative evidence of adaption and community ownership of the program and enablers and barriers to its implementation and continuation. The overall aim is to synthesise evidence of attributable impacts on student attitudes, lifestyle and behaviour and the societal (government, school and wider community) multiplier on initial investment up to and beyond two years triangulated with qualitative evidence to enable informed assessment of expected long term community impacts and returns from SAKGNP investment.

105

Table 2 Domain allocation and weightings of student and parent survey questions.a Domain Student survey Garden lifestyle behaviours Kitchen lifestyle behaviours Eating habits Food choices Parent survey Garden lifestyle behaviours Kitchen lifestyle behaviours Eating habits Food choices

Question #

# of questions

Sum of domain weights

11e18 5, 6a, 7e10 6b, 6c, 20, 22 19, 21, 23

18 17 4 5

29 31 8 8

7 5 5 4

11 10 10 7

11e17 7, 8a, 8e, 9, 10 8b, 8c, 8d, 20, 22 18, 19, 21, 23

a Student and parent surveys with all 23 questions are provided in supplementary materials.

2.1. Triangulated evidence of short term impacts on student domains of interest A modified case-control design and pre-post analysis was adopted to assess incremental impacts of the SAKGNP across the four domains of interest. An initiative group of SAKGNP schools were matched to a comparison group of schools on factors including socioeconomic levels, school size and geographic region and participating Australian states. Comparison schools were invited from a group of schools that had been accepted into the SAKGNP but were yet to commence. They formed a natural control group. A total of 14 comparison schools were invited and accepted. Initiative schools were identified from SAKGNP schools that had been running garden and kitchen classes for the longest time (a minimum of one year of classes). In matching to control schools a potential pool of 32 initiative schools were invited to participate. Email letters were sent to the principals explaining the evaluation activities. A follow up phone call to the principals was made if there was no reply to the email invitation. By the time the 28th initiative school accepted the invitation to participate all factors had been matched (Table 1). The last two schools to accept were retained as ‘stand by’ schools in case one of the initiative schools withdrew from the evaluation, while two of the invited initiative schools indicated they were unable to participate. Relevant data pertaining to attributes of schools were available on the national school database, the My School website (www. myschool.edu.au/). Geographic region was classified according to metropolitan, provincial, remote or very remote, as determined according to the Schools Geographic Location Classification Scheme of the Ministerial Council for Education, Early Childhood Development and Youth Affairs. Socio-economic level was measured by the Index of Community Socio-Educational Advantage (ICSEA) with schools above the average ICSEA score (1000) as ‘High’, and those below as ‘Low’. School size was allocated as ‘large’ (over 200 students), ‘small’ (between 51 and 199 students) and ‘very small’ (50 or fewer students). The 28 initiative and 14 comparison schools (Table 1) who agreed to participate in the evaluation provided sufficient data to match attributes between groups at a school level and sufficient

numbers of returned student (491 versus 260) and parent (300 versus 234) surveys for well powered statistical analysis to compare initiative and comparison schools across domains of interest. A pragmatic powering approach was used, which in the face of highly uncertain short and long term effectiveness and cost effectiveness of the program followed Bayesian value of information principles with respect to obtaining the most relevant data as feasible within the evaluation timeframe and budget constraint to efficiently inform government decision making allowing for relevant decision contexts (Eckermann et al., 2010). Relevant decision contexts included: (i) needing to balance attributes across schools to minimise impact of known school level confounders; (ii) obtaining adequate individual (student and parent) data within schools to address short term impact of the program on the four domains of interest; and (iii) obtaining data up to and beyond two years so as to evaluate multiplier impacts and program acceptance across schools and wider communities. To survey students and parents in relation to the four domains of interest adaptation of survey questions from the pilot SAKG evaluation in Victoria (Block and Johnson, 2009) was undertaken (see ONLINE FILE A for SAKGNP student and parent surveys). The 44 student and 21 parent survey questions were each mapped to one of the garden lifestyle behaviours, kitchen lifestyle behaviours, eating habits and food choices domains. For each domain the related SAKGNP survey questions were normalised across 3, 5 and 7 point scales used by rescaling scores between 0 (worst) and 100 (best) with linear interpolation (e.g. 5 point scales were scored 0, 25, 50, 75, 100). These normalised scores for each question were then assigned predetermined weights (see supplementary materials) reflecting their relative importance to that domain as assessed by consensus of 3 health and nutrition experts (Table 2). For example, in the garden lifestyle behaviours domain for students the question ‘How confident are you at weeding’ was assigned a weighting of one, whereas the question ‘Do you like gardening?’ was assigned a weighting of three. Hence the ‘Do you like gardening?’ question was considered to be three times as important as confidence at weeding in contributing to the garden lifestyle behaviours domain.

Table 1 Characteristics of the initiative and comparison schools. School category

Initiative N ¼ 28 Comparison N ¼ 14 Total a

Australian statea

Socio-economic level

School size

NSW

Qld

SA

Tas

WA

Metro

Geographic region Provincial

Remote

High

Low

Large

Small

Very small

9 5 14

6 1 7

8 4 12

2 1 3

3 3 6

9 4 13

17 9 26

2 1 3

11 6 17

17 8 24

13 8 21

6 4 10

9 2 11

NSW ¼ New South Wales; Qld ¼ Queensland; SA ¼ South Australia; Tas ¼ Tasmania; WA ¼ Western Australia.

106

S. Eckermann et al. / Social Science & Medicine 114 (2014) 103e112

Within the time constraints of the evaluation, linear rescaling of items to a numeric scale and within domain item weighting based on expert consensus was considered the most appropriate approach to enable pre-specified domain assessment, in the absence of fully validated instruments for student and parent surveys of the SAKGNP. Previous assessment of the SAKG program (Block and Johnson, 2009) limited analysis to comparison of responses to individual survey questions. While the range of survey questions used by Block and Johnson (2009) was maintained in the current study, survey questions were adapted to allow ordinal and scale properties necessary for domain assessment. The weighted sum of survey questions represented a numerical expression of each domain for student and parent surveys. The higher domain scores, the more positive or healthy the student was evaluated in regard to that domain. Initiative and comparison schools in the study were provided with paper surveys which were administered to Year 6 students during a classroom activity. This student survey included 44 questions in relation to the four domains of interest. An information kit was emailed to each school explaining the evaluation activity as well as including participant information sheets, consent forms and the student survey. Schools were also provided with parent surveys with 21 questions related to the four domains of interest. These surveys were administered to the parents of the Year 6 students who completed the student survey. The parent surveys were to be taken home and could be completed by a parent alone or together with the student. Domain scores between initiative and comparison schools were compared at a student level adjusting for potential confounding variables, including student gender, grade and time at school, along with their school's geographic region, socioeconomic level and size. Adjustment was undertaken including these variables as main effects in regression analysis employing multi-way ANOVA. Variables were considered significant with p < 0.05. Student-level variables were particularly important to adjust for as they could not be matched in the sample design. School-level variables were also adjusted for as matched balancing between initiative and comparison schools was somewhat disrupted with five out of 28 initiative schools and three out of 14 comparison schools not returning surveys. For example, this led to comparison schools having a higher proportion of parents with high socioeconomic levels (54.5%) than initiative schools (34.8%). A process of triangulation was employed to consider links between:  Parent survey domain differences between SAKGNP initiative versus comparison schools;  Student survey domain differences between SAKGNP initiative versus comparison schools; and  Pre-post SAKGNP summary responses across the four domains. The processes of triangulation enabled better attribution and strength of findings where there was agreement between pre-post and comparative findings. Comparative analysis addresses weaknesses of pre-post analysis in controlling for observer biases such as Hawthorne effects, as well as changing factors outside of SAKGNP (e.g. children getting older, environmental factors, other policy changes etc.). Pre-post analysis addresses weaknesses of comparative analysis in not being able to adjust for non-observed selection factors and other potential confounders between initiative and comparison schools. 2.2. Adjusting for additional potential confounders where available Potential confounding from differences in student factors such as home environment was controlled for where possible. Parent

demographic factors including country of birth, language and level of education were able to be directly adjusted for with data from the parent survey. Adjusting for education level turned out to be particularly important, as a significantly higher proportion of comparison school parents were tertiary qualified (68.1% versus 57%, p ¼ 0.010) and parents' level of education was found in regression analysis to be significantly associated with eating habits and social behaviour. Cooke et al. (2004) has also previously found a significant association between maternal education level and children's vegetable intake. 2.3. Interpreting student and parent surveys Students initially completed surveys in their schools and then took the parent surveys home to be completed with their parents (or guardians). Joint parent-student survey (dyad) responses have previously been suggested to be more reliable than students' or parents' responses alone (Ungar et al., 2006). In a joint survey approach parents are considered to act as enablers to enhance cognitive function relative to surveying children alone, while gaining access to information required for informed responses absent with surveying parents alone. In particular this addresses evidence of parents being reliable reporters for observable behaviours (for example, the expression of symptoms and physical function), but less reliable reporters of cognitive and emotion attributes (Petrou, 2003). Increased cognition relative to surveying students can enable greater scope for interval scales to be employed, which aid the sensitivity and interpretability of surveys. Linking of parent and student surveys to explore relationships between student-parent dyad surveys versus student surveys alone would have been valuable. However such linkage proved administratively difficult within the school setting, with failure to record identification numbers on surveys to match on. Devoting evaluation resources to ensure linkage between student and parent surveys is recommended to address this issue in future research. Such linkage could also explore relationships and information flows between students and parents more generally, to enrich estimation of investment multipliers with more sophisticated network analysis methods (Hawe and Ghali, 2008), as later considered in discussion. 2.4. Estimating investment multipliers Data were collected in SAKGNP initiative schools to allow direct estimates of investment multipliers up to and beyond the two year mutual obligation period. These multipliers represent the combined value of Australian Government investment, and school and volunteer activity observed over the evaluation period relative to the grant investment. An investment form was developed in Microsoft Excel to survey the grant funding, school and community staff and volunteer activities and school and wider community resources donated to the SAKGNP at initiative schools. Data included the Australian Government grant expenditure, staff specialist and program coordinator time and rates of pay, community donations and volunteer contributions and SAKGNP classes. SAKGNP expenditure by area was cross checked and calibrated for internal consistency against total SAKGNP expenditure and class time. Development of the investment form included trialling by an initiative school principal, with modifications made based on feedback. After distributing investment forms to schools, follow-up emails and telephone calls were made to school principals to clarify any data and calibration issues that arose. Once collected, investment data were analysed to: (i) calculate multiplier effects from investment by the Australian Government on

S. Eckermann et al. / Social Science & Medicine 114 (2014) 103e112

school and wider community activity; (ii) consider the school and wider community ownership and local adaptation of the SAKGNP model; and (iii) evaluate evidence on the sustainability of the SAKGNP. Evidence from some initiative schools extended beyond two years, and thus the program sustainability over time could also be considered in assessing whether, and the extent to which, garden and kitchen lessons continued beyond the two years of the capital funding and mutual obligation agreement period. These data informed whether SAKGNP initiatives at schools continue in their current form, evolve, expand or are scaled back beyond two years, and provided indicators of the long term multiplier effects, sustainability and ultimately whether these programs become accepted and integrated within communities. Longer term sustainability of the SAKGNP was also considered in relation to curriculum integration and evolution of garden and kitchen classes in schools and communities beyond two years; and comparison of the scale of classes and school and community volunteer activity beyond two years, relative to that up to two years. Analysis of multiplier impacts of Commonwealth investment on school and community activity was limited in scope to school and volunteer resources captured with interactions and activity generated at school and hence at any point is conservative to the extent only within school activity was captured. Potential wider activities engendered by the SAKGNP across community networks with students and parents in homes and networks in the wider community were not captured. However, this limitation may not necessarily be conservative in assessing dynamic impacts over time as such impacts can go up and down or more generally be nonlinear in impact and rely on density of key relationships with, and status of, gatekeepers or champions (Hawe et al., 2009) and long term community ownership and embedding of the SAKGNP. We will later return to consider such issues in discussion of the potential to enrich analysis with use of more sophisticated network methods than multipliers. Nevertheless, while more sophisticated network analysis methods were beyond the resources of our analysis, qualitative analysis was employed toward the same ends in enriching consideration of evidence from investment multipliers. Qualitative consideration of enablers and barriers to schools and communities in continuing garden and kitchen programs and lessons beyond the two years of the agreements associated with initial capital investment were evaluated in survey and interview settings and jointly assessed in triangulation with evidence of multipliers over time. In particular, qualitative analysis included consideration of enablers and barriers to the long term viability of running the SAKGNP or local adaptation of the model in being self-funded beyond the two year capital funding agreements. The economic data and its analysis also provided a basis for assessing the financial implications of sustaining the SAKGNP in schools beyond grant funding mutual obligation periods. Such data and analysis are critical to inform government decision-making, particularly during periods of tight budget allocations or financial austerity. Triangulation of this evidence with qualitative assessment can also aid in assessing the integration of program support in systems and maintaining of relationships such as that between staff and volunteers (intensiveness). 3. Results 3.1. SAKGNP impacts on student behaviour in four domains of interest There were a number of survey domains and questions where the SAKGNP demonstrated significant impacts in triangulation of comparative student and parent surveys with pre-post analysis,

107

which are summarised below (for detailed results, refer to Yeatman et al. (2013)). 3.1.1. Kitchen lifestyle behaviour domain Comparisons between initiative and comparison schools from parent surveys showed children from initiative schools liked cooking more and helped with cooking more than children from comparison schools, and that parents liked cooking with their child more. Nearly 20% of parents indicated that they prepared more meals at home after their children participated in the SAKGNP; 77.4% of parents indicated their child asked them to make foods that had been made at school as part of the SAKGNP; and 71.9% of parents reported that their child was more willing to cook at home since the start of the SAKGNP. There was a significant increase in the overall domain score for kitchen lifestyle behaviours for initiative compared to comparison schools (t ¼ 2.35, p ¼ 0.019) after adjusting for potential confounding factors. Mothers of students were found to report significantly higher kitchen lifestyle domain scores than fathers (t ¼ 2.11, p ¼ 0.036), and provincial schools had significantly higher scores than metropolitan schools (t ¼ 2.80, p ¼ 0.005). Hence, parent survey findings support a significant impact of the SAKGNP on kitchen lifestyle behaviours in students and the reliability and sensitivity of this domain. These findings are further supported by self-reported retrospective pre-post analysis in initiative schools, where 18.6% (54/290) of students reported preparing their own meals at home more often and only 1.4% (4/290) less often. Triangulation therefore strongly supports the SAKGNP improving kitchen lifestyle behaviours. 3.1.2. Food choices domain Students in initiative schools reported greater willingness to try new foods (generally and if they had cooked it or grown it). There was a significant increase in student overall domain scores for food choices in initiative compared to comparison schools (t ¼ 2.26, p ¼ 0.024), after adjusting for potential confounding factors. Female students were found to have significantly higher scores than male students (t ¼ 4.54, p < 0.001), and students from provincial schools had significantly higher scores than students in metropolitan schools (t ¼ 3.74, p < 0.001). Hence student survey findings support a significant impact of the SAKGNP on food choices of students and the reliability and sensitivity of this domain. This set of findings is further supported by self-reported retrospective pre-post survey analysis of students in initiative schools, where 20.0% (58/290) reported eating fruit and vegetables more often and only 0.7% students (2/290) less often. Triangulation between comparative student and pre-post analysis therefore supports an improvement in food choices attributable to the SAKGNP. 3.1.3. Eating habits and behaviours Interestingly, despite improving food choices, there was a nonsignificant trend towards students reporting lower scores for eating habits and behaviours (t ¼ 1.90, p ¼ 0.058) in the initiative schools compared to comparison schools, after adjusting for potential confounders. This trend was observed to a lesser degree in parent surveys where a non-significant difference in domain scores for eating habits was observed between initiative and comparison schools (t ¼ 1.10, p ¼ 0.273) after adjusting for confounding factors. A statistically significant effect of parent education was found for this domain (t ¼ 2.40, p ¼ 0.017) with students of tertiaryqualified parents having higher scores than students whose parents did not finish high school. Also, there was evidence of a significant effect due to school size, with parents of children from very

108

S. Eckermann et al. / Social Science & Medicine 114 (2014) 103e112

small schools having higher scores than large schools (t ¼ 2.48, p ¼ 0.014). However, the potential for a trend in eating habits and behaviours from adjusted comparative analysis of parent and student surveys was not supported by pre-post analysis, where 15.6% (45/ 289) reported eating meals together more often and only 0.3% (1/ 289) less often after the SAKGNP. Hence the potential for a trend with comparative analysis between initiative and comparison schools may be by chance or reflect differences in unobserved school confounders, environmental or parent factors. Alternatively, given findings of significantly improved food choices and awareness, a negative trend in self-reported eating habits and behaviours in initiative schools compared to comparison school could be explained by the SAKGNP leading to more accurate reporting of food consumption. Overall, triangulation of evidence does not support the SAKGNP having a positive or negative short-term impact on eating habits and behaviours with positive evidence from pre-post analysis not supported by negative trends in comparative analysis. 3.1.4. Garden lifestyle behaviour domain For the gardening lifestyle behaviours domain pre-post analysis of household activity suggests that the student's involvement in the SAKGNP led to 32.3% (91/278) helping more in the garden and only 3.6% less (10/278). However, there were no significant differences or trends for the garden lifestyle behaviour domain in comparative analysis in the student or parent survey findings across domains. Overall, triangulation of evidence is inconclusive as to whether the SAKGNP has a positive short term impact on garden lifestyle behaviours, with positive pre-post analysis not supported in comparative analysis. In summary, triangulated pre-post and comparative analysis for the four domains of interest shows the SAKGNP had significant attributable positive impacts on food choices and kitchen lifestyle behaviour domains in students, while impacts on garden lifestyle behaviour and eating habit domains were inconclusive. 3.2. Investment, activity and investment multipliers up to and beyond two years All 23 of the initiative schools who returned parent and student surveys and one other school who did not complete student or parent surveys agreed to undertake the investment and activity survey, while 15 of those 24 (62.5%) returned the investment form surveys. There were significant differences between the characteristics (size, region, SES) of the 15 schools that completed the investment form in comparison to the nine schools those that did not, as shown in Table 3. The schools that did not respond were more likely to be rural, smaller, low SES and in New South Wales. Nevertheless the response rate (62.5% of schools) was relatively high. The schools who did respond represented a larger proportion of student survey responses (410/491 or 83.5%) and parent survey responses (246/ 300 or 84.7%) and data were provided for all major variables for analysis in all those schools who did respond. Table 4 summarises the average expenditures per school. The average Australian Government SAKGNP grant for capital expenditure provided was $44,758, with approximately two thirds (66.2%) of that expenditure on kitchen capital and one third (33.8%) on garden capital. Kitchen staff expenditure ($29,428) contributed a significantly higher proportion (60.3%) of specialist staff costs than garden staff expenditure ($19,397 or 39.7%) up until two years. The 51.7% higher expenditure on kitchen relative to garden staff specialists was

Table 3 School investment survey responders. Geographical location

Metropolitan Provincial Remote Total Socioeconomic High SES Low SES Total School size Large Small Very small Total State NSW QLD SA TAS WA Total

Schools which returned investment forms (N ¼ 15)

Schools which did not return investment forms (N ¼ 9)

#

%

#

%

7 7 1 15 level 7 8 15

46.7% 46.7% 6.7% 100.0%

1 7 1 9

11.1% 77.8% 11.1% 100.0%

46.7% 53.3% 100.0%

1 8 9

11.1% 88.9% 100.0%

10 2 3 15

66.7% 13.3% 20.0% 100.0%

1 3 5 9

11.1% 33.3% 55.6% 100.0%

3 3 4 2 3 15

20.0% 20.0% 26.7% 13.3% 20.0% 100.0%

5 1 3 0 0 9

55.6% 11.1% 33.3% 0.0% 0.0% 100.0%

Table 4 Average per school Australian Government capital grants and school expenditure on SAKGNP up to two years (SAKGNP agreement period) by type (AU$). Expenditure Australian government capital grant investment Recurrent staff specialist expenditure Program co-ordinator expenditurea Total school staff expenditure Maintenance and disposables Total school expenditure Total (school and grant expenditure)

Total

Garden

Kitchen

$44,757.67

$15,147.40

$29,610.26

$48,824.36

$19,396.70

$29,427.66

$6967.83

$2768.15

$4199.69

$55,792.19 $15,696.21 $71,488.41 $116,246.07

$22,164.85 $5388.32 $27,553.17 $42,700.57

$33,627.35 $10,307.89 $43,935.24 $73,545.51

a Program co-ordinator expenditure allocated to garden and kitchen using the same proportions as specialist staff expenditure.

explained by an 11.8% higher wage rate per hour including on-costs ($39.72 versus $35.53) and 35.7% greater hours (detailed in Table 5). The average expenditure per year on staff specialists up to two years including on-costs was $10,011 for garden specialists and $15,221 for kitchen specialists with an average 282 h of garden specialist time and 383 h of kitchen specialist time.

Table 5 Average wage rate and paid hours per school year for SAKGNP staff specialists up to two years. Specialist staff costs up to two years (AU$)

Total

Garden

Kitchen

Average hours of staff specialist time per school Average wage rate per hour Staff specialist expenditure up to two years Per yeara Hours of staff specialist time Staff specialist expenditure up to two years

1286.8

545.9

740.9

$37.94 $48,824.36

$35.53 $19,396.70

$39.72 $29,427.66

664.5 $25,210.51

281.9 $10,015.51

382.6 $15,195.01

a Schools were on average observed for 1.94 years up to two years, with a total of 29.05 years observed for 15 schools and 2/15 schools having less than two years observed.

S. Eckermann et al. / Social Science & Medicine 114 (2014) 103e112

Table 6 presents the breakdown of reported specialist staff costs. The garden and kitchen specialists, on average, ran 159 h of SAKGNP garden lessons and 250 h of SAKGNP kitchen lessons per year, respectively. Hence, the staff specialist average costs per hour of lessons including on-costs were $63.18 per hour for garden lessons and $61.26 per hour for kitchen lessons. Costs associated with SAKGNP coordination suggest that these SAKGNP specialist staff cost per hour of classes would be increased by 14% to $72.19 and $70.00 per hour for the garden and kitchen SAKGNP classes, respectively. From staff specialist and lesson hours data it also can be inferred that, up to two years, for the garden and kitchen specialists the ratio of hours worked to lesson contact hours is 1.78: 1 and 1.54: 1, respectively. This explains the almost equivalent specialist staff cost of garden or kitchen classes per hour, as the 12% higher cost per hour of kitchen staff is more than compensated for by a lower number of paid preparation hours per hour of garden classes. Importantly, these costs per class hour and ratios of total relative to contact hours are also respectively comparable to and less than the minimum of those typically found in conventional contemporary Australian primary school classroom settings. Typically in primary schools there is a maximum of 21 h 50 min class hours per week and 37 paid hours and hence a minimum ratio of hours paid to class hours of 1.69 (http://www.det.wa.edu.au/teachingwa/ detcms/navigation/working-in-a-public-school/work-life-balance/ ; Australian Bureau of Statistics, 2006). 3.2.1. Community volunteer contributions e time and donations Community volunteers on average contributed 2641 h per school up to two years, with 1351 h in the garden and 1289 h in the kitchen (Table 7). This is equivalent to an average contribution for each school of 698 h per year to the SAKGNP gardens and 666 h per year to the SAKGNP kitchens, or a total of 1364 h per year. This represents a multiplier of more than twice (2.05 times) the number of garden and kitchen specialist hours per year up to two years. In addition, there was an average donated community capital and disposables of $11,252 in schools up to two years, or $5815 per year over the average 1.94 years observed per school. Hence from an Australian Government perspective, the multiplier from their average capital investment of $44,757 in the SAKGNP up until two years (average 1.94 years observed) includes average school expenditure on the SAKGNP of $71,488, average community donated capital and disposables of $11,252 and 2641 h of community volunteer time. Valuing community volunteer time at the same rate per hour as garden and kitchen staff specialists the Australian Government multiplier on grant investment of $44,757 up to two years, from observed school and community (volunteer and donations) activity is 5.07, with an equivalent value of $226,736 ($71,488 or 1.60 of the multiplier attributable to additional school activity and $110,491 or 2.47 from the value of volunteer time and community donations). That is, initial Australian Government

Table 6 Average number of garden and kitchen class hours per school, per year and associated staff costs per class up to two years. Decomposing specialist staff costs (AU$)

Total

Garden

Kitchen

Average class hours per school to 2 years Per year average class hoursa Specialist staff hours per hour of classes Staff specialist cost per class SAKGNP staff costs per class including program co-ordinator costs

787.4 409.40

307.0 159.22 1.778 $63.18 $72.19

480.4 250.18 1.542 $61.26 $70.00

a

Schools were on average observed for 1.94 years up to two years, with a total of 29.05 years observed for 15 schools, with 2/15 schools having less than two years observed.

109

Table 7 Average per school community SAKGNP volunteer time and capital and disposable donations up to two years (SAKGNP agreement period). Activity (hours) and value (AU$)

Total

Garden

Kitchen

Average hours of community donated time per school up to two years Ratio of volunteer to staff specialist hours Value of community volunteer timea Donated capital Total community contribution up to two years Total school contributions Total grant contributions Total (volunteer, school and SAKGNP Aust. Govt. funding contributions)

2641.1

1351.7

1289.4

2.052

2.476

1.740

$99,238.49 $48,029.24 $51,209.24 $11,252.44 $6991.70 $4260.75 $110,490.93 $55,020.94 $55,469.99 $71,488.41 $27,553.17 $43,935.24 $44,757.67 $15,147.40 $29,610.26 $226,737.01 $97,721.51 $129,015.50

a Volunteer time is valued using the same rate per hour as garden and kitchen staff specialist consistent with the replacement value method for volunteers recommended by Salamon et al. (2011).

investment of $44,757 leads to 4.07 times the additional value ($181, 979) in direct school and community activity up until two years and a multiplier of 5.07 including the initial investment. 3.2.2. Impacts beyond two years Considering evidence beyond two years, on average schools were observed for 2.40 years since starting the SAKGNP, 0.46 years greater than the average 1.94 years observed across schools up to two years (Table 8). From an Australian Government perspective the multiplier increases to 5.91 at 2.4 years, with a value of school ($87,289) and volunteer contributions ($132,376) leading to a total of $264,423 of overall activity, relative to a $44,757 initial investment. In the eight schools that had more than two years of time since starting the SAKGNP, garden and kitchen classes continued in all of them. In seven of these eight schools SAKGNP staff specialists continued to run classes, while in one school teachers ran the classes. Across these eight schools the average hours of garden and kitchen lessons per year was 165 and 205 during the first two years and increased to 182 and 253 h per year beyond two years, respectively. This represented a 10% increase in hours of garden classes per year and 23% increase in hours of kitchen classes per year. Overall total garden and kitchen class hours per year increased from 371 to 436, a 17% increase. This provides clear evidence of schools continuing classes and extending the SAKGNP. This also provides evidence of the program evolving locally to have a greater proportion of kitchen relative to garden class hours. Volunteer hours per year amongst the eight schools observed beyond two years were 663 in gardens and 683 in kitchens during Table 8 Investment multiplier for the value of school, community and grant contributions relative to Australian Government grant investment (for schools who ran SAKGNP beyond two years, N ¼ 8). Value of activity (AU$)

Total

Garden

Kitchen

Value of community volunteer timea Donated capital Total community contributions Total school contributions Total grant contributions Total (volunteer, school and SAKGNP Aust. Govt. funding contributions ) Multiplier

$120,607.06

$57,955.77

$62,651.29

$11,769.11 $132,376.17 $87,289.93 $44,757.67 $264,423.77

$7345.03 $65,300.80 $34,022.73 $15,147.40 $114,470.94

$4424.08 $67,075.37 $53,267.19 $29,610.26 $149,952.83

a

$264,423.77/$44,757 ¼ 5.91

Volunteer time is valued using the same rate per hour as garden and kitchen staff specialist in determining the multiplier of school and community activity relative to Australian Government investment under the SAKGNP.

110

S. Eckermann et al. / Social Science & Medicine 114 (2014) 103e112

the first two years, which respectively decreased and increased to 565 and 745 per year beyond two years. Hence, volunteer hours reduced by 15% per year in the garden while kitchen hours increased by 11%. Combined, total volunteer hours per year decreased marginally by 3% from 1346 to 1310 h per year. 4. Discussion Analysis of the overall impact of the SAKGNP has shown that across the four domains of interest, triangulation of evidence leads to the conclusion that the SAKGNP can be attributed with significantly improving kitchen lifestyle behaviours and food choices. This provides a strong signal that as a health promotion and prevention program the SAKGNP works in achieving stated objectives in terms of attitude changes to food choices and cooking, and the development of cooking skills. Investment analysis provides evidence that there is a large multiplier impact from the initial Australian Government capital funding of the SAKGNP on school and community activity, particularly community volunteer involvement. The 5.07 multiplier with the SAKGNP compares with a multiplier from government investment of 2.93 observed in the Victorian pilot SAKG program evaluation over two years (Block and Johnson, 2009), where schools undertaking the Victorian SAKG pilot for every $1 of government funding invested generated $1.93 of additional resources ($0.43 cash donations, $0.39 donated goods and services and $1.11 of donated labour time). However, it should be noted that the Victorian government invested an average of $72,080 per school; $38,984 in the initial establishment year and $33,096 in the implementation year, which contributed towards recurrent as well as capital expenditure. Hence, the multiplier up to two years estimated in the Victorian SAKG pilot would have been expected to be higher if initial investment had not included recurrent as well as capital expenditure grants. The high investment multiplier in the SAKGNP provides a strong indication that the SAKGNP is effective as a health promotion and prevention program in engaging with school and community networks and building social capital to enable cultural embedding and community ownership of primary health promotion and disease prevention strategies (Hawe and Shiell, 2000; Moore et al., 2006). The continuation of garden and kitchen classes with somewhat increased classes and more emphasis on kitchen rather than garden beyond two years provides further indication of the successful evolution and integration in school and community networks, school and community valuing of the program ownership, and the sustainability of the SAKGNP. 4.1. Additional network analysis Use of more sophisticated network analysis methods for complex systems, as described by Hawe et al.(2008; 2004b; 2009; 2004a), including considering pre- and post- network density (number of linkages per individual) or the extent of champion and gatekeeper involvement, could further enrich and strengthen such findings. In the case of the SAKGNP such network analysis could be undertaken within and between specialist garden and kitchen staff, curriculum integrated and wider school staff and volunteers, students and parents. Linkage to impacts on wider community systems, networks and events such as building of other community gardens and kitchens, that social capital building inherent in the SAKGNP may help enable, could also be considered. Evaluating multiplier effects over time and triangulation with qualitative evidence from surveys of barriers and enablers in this study may be somewhat cruder than these network methods in allowing for impacts on complex systems and may fail to capture

the full extent and density of those impacts. Nevertheless, we suggest that the investment multiplier considered over time, and particularly beyond obligation period, combined with qualitative evidence provides a strong platform to both triangulate with, and direct the best use of, such more sophisticated methods. In considering the long term effects one should remember that dynamic impacts of health promotion and prevention strategies on networks in complex systems arise over time, where feedback loops between program and network building aspects are not necessarily linear and can result in unintended consequences, as Hawe et al. (2009) emphasise in making a strong case for use of more sophisticated network methods. In the case of the SAKGNP one such unintended consequence could be the crowding out of other community gardens. Comparison with matched natural comparator schools yet to undertake the SAKGNP, while allowing a strong design in many respects, does preclude the potential for comparatively evaluating such potential for crowding out. Nonetheless, one should remember that while crowding out may be an issue when there are limited resources for undertaking programs such as the SAKGNP, the SAKGNP itself aims to address these in building social capital, improving skill sets and capacity building for kitchens and gardens and the improving of their use. Hence, while potential for some crowding out may be a short term consideration, programs such as the SAKGNP if successful build skills, networks and social capital themselves to enable greater capacity (Hawe and Shiell, 2000; Hawe et al., 2009); the ability to build further community gardens and kitchens. Most importantly, in assessing health promotion and prevention strategies such as the SAKGNP in complex systems such as schools there is a developing toolkit of approaches which can quantitatively as well as qualitatively capture the community network impacts, ownership and embedding of programs which are critical to their continuation, success and long term return on investment. 4.2. Comparison with conventional modelled cost effectiveness evaluation An alternative, conventional modelled health economic assessment and interpretation of the SAKGNP could be considered, with expected outcomes modelled, based on the observed impacts and compared with the total resources devoted to the SAKGNP. However, in attempting to model long term impacts critical for robust assessment of health promotion and prevention programs it should be clear that they would be critically dependent on the ownership by the school and wider community of the SAKGNP, in addition to attitude and behavioural changes associated with the SAKGNP. That is, modelling a health promotion and prevention strategy delivered in a complex school setting, where the integration with school and community networks, building of social capital at a community level and reaching a community threshold level of where healthy behavioural change takes place would be the key drivers (Shiell et al., 2008). In order for such a model to reliably predict the long term outcomes of the SAKGNP and health system related (and other public sector) cost savings that might ensue, it principally needs to consider the community level ownership, impact and building of social capital in translating the attitude and behavioural impacts observed to final outcome and cost implications. Consequently, it should be clear that with net positive impacts on attitudes and behaviour established with the SAKGNP, the multiplier observed in school and community activity provides a good indicator of whether the SAKGNP is expected to be successful. Community volunteer time should be fully valued in calculating the multiplier, given that the multiplier is calculated to indicate community networking, acceptance and valuing of the program. Hence, valuing volunteer time equally to that of SAKGNP staff is

S. Eckermann et al. / Social Science & Medicine 114 (2014) 103e112

appropriate. This also aligns with the replacement value method for volunteers recommended by Salamon et al. (2011) as more appropriate in assessing economic value of volunteers than opportunity cost of volunteers time or contingent valuation of their time (willingness to pay). The long term success of the SAKGNP in terms of health outcomes and health system costs also clearly depends on the integration of garden and kitchen lessons into the school curriculum. If such integration can be successfully undertaken, the long term attributable health impacts and downstream health system cost savings at a community level are expected to be significant, given current literature on impacts of community involvement and attitude and behaviour changes at a community level. For example, positive impacts have been shown on healthy eating from children developing skills to cook their own food (Walters and Stacey, 2009) and knowledge, attitudes and fruit and vegetable consumption from growing their own food, particularly with community gardening in schools (Libman, 2007; McAleese and Rankin, 2007; Morgan et al., 2010; Morris et al., 2001; Robinson-O'Brien et al., 2009; Somerset and Markwell, 2008). This second set of findings has also been supported more generally with community gardening in adults, where for example Litt et al. (2011) showed that community gardeners have a significantly higher vegetable and fruit consumption (5.9 per day) than other gardeners (4.6 per day) or non-gardeners (3.9 per day). Zick et al. (2013) have also shown that community gardeners have a lower BMI, thus providing an important strategy to address community overweight and obesity. Hence, long term integration of food gardens into the school curriculum can be expected to have health impacts both directly for students but also indirectly in the wider community, given student and community skills and social capital are built by the SAKGNP over the long term. 4.2.1. SAKGNP challenges for the future Challenges remain to provide a process of full integration of the SAKGNP into schools' curricula. Qualitative evidence points to the SAKGNP working where a whole school approach is adopted, specialist garden and kitchen staff are employed and there is sharing of food. Perceived barriers to the feasibility and continuation of the SAKGNP and its impacts have been indicated in national workshop discussions, school visits and stakeholder interviews (Yeatman et al., 2013) as primarily related to the continuing need of schools to raise funds to support garden and kitchen specialists and the need to integrate the SAKGNP using a whole of school approach. Continuity of staff running garden and kitchen classes, and their impact in continuing connection with volunteers, are also related issues. Several options emerged for the integration and evolution of the SAKGNP into the primary school curriculum which would address these perceived barriers, noting that the SAKGNP can contribute to school classes while having health promotion and prevention objectives. They include continuing with staff specialists and other teachers in SAKGNP classes being additionally funded as part of school curriculum, and the potentially budget neutral option of running SAKGNP classes employing teachers who have skills to run garden and kitchen classes. This last approach has already been taken up in evolution of the SAKGNP at one of the eight schools observed beyond the two year agreement period. This provides further indication of the integration of garden and kitchen classes in the school community and curriculum. As a natural experiment, these SAKGNP schools could be followed for a longer time to gather more evidence in relation to the evolution and relative success of alternative approaches. If staff specialists were running such classes independently, the analysis of staff costs and paid hours to contact hours points to the potential to incorporate the SAKGNP into lesson time for students

111

without increasing recurrent expenditure. This would be possible if staff specialists were fully integrated as part of the curriculum. However, schools had a schoolteacher present in 84% of garden classes and 81% of kitchen classes in the SAKGNP up to two years alongside the SAKGNP staff specialist, while volunteers were additionally present in 58% of garden classes and 77% of kitchen classes. The participation of school teachers in the SAKGNP classes observed up to two years highlights that modification to the classroom model under the SAKGNP would need to take place for the SAKGNP to be integrated into the curriculum without the need for additional recurrent school expenditure or fundraising. Options such as employing teachers with specialist garden and/or kitchen skills to run these classes could be fruitful to consider in this regard. Such a model could provide a natural mechanism for long-term viability of schools running SAKGNP classes, without the pressure for fundraising to support garden and kitchen specialist staff. Alternatively, the SAKGNP could be integrated into state curriculum with additional recurrent staff expenditure supported by curriculum based funding. Such curriculum integration would provide greater security and continuity of employment for garden and kitchen staff specialists and address garden and kitchen staff retention, another major perceived barrier to long-term viability of the SAKGNP in schools. Continuity of staff specialists was also seen as important in retaining continuity in contact with community volunteers particularly as well as staff, students and the wider community. 5. Conclusions The SAKGNP has been successful as a health promotion and health care prevention strategy in terms of stated objectives for kitchen lifestyle behaviours and food choice domains and in implementation of the SAKGNP in school and community settings. A high multiplier impact estimated as 5.07 over a two year mutual obligation period (rising to 5.91 over an average of 2.4 years) indicates the high level of program acceptance by school and local community networks. Schools that continued garden and kitchen classes beyond the two year agreement period on average scaled up the programs classes by 17% and there was evidence of local evolution of the SAKGNP with an increased proportion of kitchen classes and one school fully integrating classes into the curriculum. The demonstrated statistically significant improvement in kitchen lifestyle behaviour and food choice domains attributable to the SAKGNP, successful integration in school and the wider community networks (reflected in high multipliers on initial government capital investment), combined with current literature, points to longer term health impacts and potential for associated cost savings provided the SAKGNP can be integrated into the curriculum. It is suggested that future studies of this kind use more sophisticated network analysis methods to qualitatively and quantitatively assess: impacts on density of interactions between all actors, the dynamic impacts on actors' status, roles as gatekeepers, enablers or champions of the SAKGNP, and the potential for wider impacts on community events and activities, for example assessing social capital and capacity building. Exploring alternative ways of integrating and evolving the SAKGNP to become part of the school curriculum is suggested, to remove the heavy reliance on school fundraising to pay specialist staff to support continuity of garden and kitchen classes and to maintain positive relationships with community volunteers. Options for integration beyond the two year funding period include full curriculum integration between garden and kitchen staff specialists and classroom teachers in garden and kitchen classes,

112

S. Eckermann et al. / Social Science & Medicine 114 (2014) 103e112

modifying the SAKGNP to have teachers with garden and kitchen skills or hybrid combinations of these approaches across garden and kitchen activities. Such changes are already occurring in some schools and could be followed as natural experiments to inform such decisions and policy for integration and evolution to local setting beyond two years. Acknowledgements The funding to conduct the national evaluation was provided by the Australian Government Department of Health and Ageing. The authors acknowledge that the evaluation would not have been possible without the contributions and cooperation of a number of groups. In particular we would like to thank the schools and school staff involved in the evaluation of the Stephanie Alexander Kitchen Garden National Program, the Stephanie Alexander Kitchen Garden Foundation, members of the Evaluation Reference Group, and participating representatives of Australian state and territory education and health departments. The support of staff within the Healthy Living Branch, Population Health Division (Department of Health and Ageing) is also gratefully acknowledged. Appendix A. Supplementary data Supplementary data related to this article can be found at http:// dx.doi.org/10.1016/j.socscimed.2014.05.056. References Australian Bureau of Statistics, 2006. Education & Work: School Teachers. Retrieved 1 April 2013, from: http://www.abs.gov.au/ausstats/[email protected]/2f762f958454 17aeca25706c00834efa/557e80a2b6d29244ca2570ec001b193d!OpenDocument. Block, K., Gibbs, L., Staiger, P.K., Gold, L., Johnson, B., MacFarlane, S., et al., 2012. Growing community: the impact of the Stephanie Alexander Kitchen Garden Program on the social and learning environment in primary schools. Health Educ. Behav. 39 (4), 419e432. Block, K., Johnson, B., 2009. Evaluation of the Stephanie Alexander Kitchen Garden Program: Final Report. Melbourne University, Deakin University, Melbourne. Cooke, L.J., Wardle, J., Gibson, E.L., Sapochnik, M., Sheiham, A., Lawson, M., 2004. Demographic, familial and trait predictors of fruit and vegetable consumption by pre-school children. Public Health Nutr. 7 (2), 295e302. Dawber, T.R., Moore, F.E.J., Mann, G.V., 1957. Coronary heart disease in the Framingham Study. Am. J. Public Health 47 (2), 4e25. de Salazar, L., Jackson, S., Shiell, A., Rice, M., 2007. Guide to Economic Evaluation in Health Promotion. Pan American Health Organization, Regional Office of the World Health Organization, Washington. Eakin, E., Lawler, S., Vandelanotte, C., Owen, N., 2007. Telephone interventions for physical activity and dietary behavior change: a systematic review. Am. J. Prev. Med. 32 (5), 419e434. Eckermann, S., Karnon, J., Willan, A.R., 2010. The value of information: best informing research design and prioritization using current methods. Pharmacoeconomics 28 (9), 699e709. Gibbs, L., Staiger, P.K., Johnson, B., Block, K., Macfarlane, S., Gold, L., et al., 2013a. Expanding children's food experiences: the impact of a school-based kitchen garden program. J. Nutr. Educ. Behav. 24 (2), 137e147. Gibbs, L., Staiger, P.K., Townsend, M., Macfarlane, S., Gold, L., Block, K., et al., 2013b. Methodology for the evaluation of the Stephanie Alexander Kitchen Garden program. Health Promot. J. Aust. 24 (1), 32e43. Hawe, P., Ghali, L., 2008. Use of social network analysis to map the social relationships of staff and teachers at school. Health Educ. Res. 23 (1), 62e69. Hawe, P., Shiell, A., 2000. Social capital and health promotion: a review. Soc. Sci. Med. 51 (6), 871e885. Hawe, P., Shiell, A., Riley, T., 2004b. Complex interventions: how far ‘out of control’ should a randomised controlled trial be? Br. Med. J. 328, 1561e1563.

Hawe, P., Shiell, A., Riley, T., 2009. Theorising interventions as events in systems. Am. J. Community Psychol. 43 (3e4), 267e276. Hawe, P., Shiell, A., Riley, T., Gold, L., 2004a. Methods for exploring implementation variation and local context within a cluster randomised trial. J. Epidemiol. Community Health 58, 788e793. Hill, S.E., Blakely, T.A., Fawcett, J.M., Howden-Chaman, P., 2005. Could mainstream anti-smoking programs increase inequalities in tobacco use? New Zealand data from 1981e96. Aust. N. Z. J. Public Health 29 (3), 279e288. Libman, K., 2007. Growing youth growing food: how vegetable gardening influences young people's food consciousness and eating habits. Appl. Environ. Educ. Commun. 6 (1), 87e95. Litt, J.S., Soobader, M.-J., Turbin, M.S., Hale, J.W., Buchenau, M., Marshall, J.A., 2011. The influence of social involvement, neighborhood aesthetics, and community garden participation on fruit and vegetable consumption. Am. J. Public Health 101 (8), 1466e1473. McAleese, J.D., Rankin, L.L., 2007. Garden-based nutrition education affects fruit and vegetable consumption in sixth-grade adolescents. J. Am. Diet. Assoc. 107 (4), 662e665. Moore, S., Haines, V., Hawe, P., Shiell, A., 2006. Lost in translation: a genealogy of the “social capital” concept in public health. J. Epidemiol. Community Health 60 (8), 729e734. Morgan, P.J., Warren, J.M., Lubans, D.R., Saunders, K.L., Quick, G.I., Collins, C.E., 2010. The impact of nutrition education with and without a school garden on knowledge, vegetable intake and preferences and quality of school life among primary-school students. Public Health Nutr. 13 (11), 1931e1940. Morris, J.L., Neustadter, A., Zidenberg-Cherr, S., 2001. First-grade gardeners more likely to taste vegetables. Calif. Agric. 55 (1), 43e46. Petrou, S., 2003. Methodological issues raised by preference based approaches to measuring the health status of children. Health Econ. 12 (8), 697e702. Robinson-O'Brien, R., Story, M., Heim, S., 2009. Impact of garden-based youth nutrition intervention programs: a review. J. Am. Diet. Assoc. 109 (2), 273e280. Salamon, L.M., Sokolowski, S.W., Haddock, M.A., 2011. Measuring the economic value of volunteer work globally: concepts, estimates, and a roadmap to the future. Ann. Public Coop. Econ. 82 (3), 217e252. Schensul, J.J., 2005. Sustainability in HIV prevention research. In: Trickett, E.J., Pequegnat, W. (Eds.), Community Interventions and AIDS. Oxford University Press, Oxford. Shiell, A., Hawe, P., 1995. Health promotion community development and the tyranny of individualism. Health Econ. 5 (3), 241e247. Shiell, A., Hawe, P., Gold, L., 2008. Complex interventions or complex systems? Implications for health economic evaluation. Br. Med. J. 336 (7656), 1281e1283. Somerset, S., Markwell, K., 2008. Impact of a school-based food garden on attitudes and identification skills regarding vegetables and fruit: a 12 month intervention trial. Public Health Nutr. 12 (2), 214e222. Stephanie Alexander Kitchen Garden Foundation, 2011. Stephanie Alexander Kitchen Garden Foundation Philosophy. Retrieved 1 October 2011, from: http:// www.kitchengardenfoundation.org.au/about-the-program/our-philosophy. Truett, J., Cornfield, J., Kannel, W.B., 1967. A multivariate analysis of the risk of coronary heart disease in Framingham. J. Chronic Dis. 20, 511e524. Ungar, W.J., Mirabelli, C., Cousins, M., Boydell, K.M., 2006. A qualitative analysis of a dyad approach to health-related quality of life measurement in children with asthma. Soc. Sci. Med. 63 (9), 2354e2366. Walters, L.M., Stacey, J.E., 2009. Focus on food: development of the cooking with kids experiential nutrition education curriculum. J. Nutr. Educ. Behav. 41 (5), 371e373. Yeatman, H., Quinsey, K., Dawber, J., Nielsen, W., Condon-Paoloni, D., Eckermann, S., et al., 2013. Stephanie Alexander Kitchen Garden National Program Evaluation: Final Report. University of Wollongong: Centre for Health Service Development, Australian Health Services Research Institute. Yeatman, H., Quinsey, K., Dawber, J., Nielsen, W., Condon-Paoloni, D., Eckermann, S., Morris, D., Grootemaat, P., & Fildes, D. Combining Realism with Rigour: Evaluation of a National Kitchen Garden Program in Australian Primary Schools. Evaluation Journal of Australasia. (in press). Yin, R.K., 1979. Changing Urban Bureaucracies: How New Practices Become Routinized. Lexington Books, Lexington, Mass. Zaza, S., Briss, P.A., Harris, K.W., 2005. The Guide to Community Preventive Services. What Works to Promote Health? Oxford University Press, New York. Zick, C.D., Smith, K.R., Kowaleski-Jones, L., Uno, C., Merrill, B.J., 2013. Harvesting more than vegetables: the potential weight control benefits of community gardening. Am. J. Public Health 103 (6), 1110e1115.