To assess from a societal perspective the incremental cost-effectiveness of the Walking School Bus (WSB) program for Australian primary school children as an obesity prevention measure. The intervention was modelled as part of the ACE-Obesity study, which evaluated, using consistent methods, thirteen interventions targeting unhealthy weight gain in Australian children and adolescents.
A logic pathway was used to model the effects on body mass index [BMI] and disability-adjusted life years [DALYs] of the Victorian WSB program if applied throughout Australia. Cost offsets and DALY benefits were modelled until the eligible cohort reached 100 years of age or death. The reference year was 2001. Second stage filter criteria ('equity', 'strength of evidence', 'acceptability', feasibility', sustainability' and 'side-effects') were assessed to incorporate additional factors that impact on resource allocation decisions.
The modelled intervention reached 7,840 children aged 5 to 7 years and cost $AUD22.8M ($16.6M; $30.9M). This resulted in an incremental saving of 30 DALYs (7:104) and a net cost per DALY saved of $AUD0.76M ($0.23M; $3.32M). The evidence base was judged as 'weak' as there are no data available documenting the increase in the number of children walking due to the intervention. The high costs of the current approach may limit sustainability.
Under current modelling assumptions, the WSB program is not an effective or cost-effective measure to reduce childhood obesity. The attribution of some costs to non-obesity objectives (reduced traffic congestion and air pollution etc.) is justified to emphasise the other possible benefits. The program's cost-effectiveness would be improved by more comprehensive implementation within current infrastructure arrangements. The importance of active transport to school suggests that improvements in WSB or its variants need to be developed and fully evaluated.
Active transport, to and from school, offers an opportunity for school aged children to increase their levels of daily physical activity. This can lead to physical, mental health, safety and environmental benefits for both the community and participating individuals . While the majority of Australian primary school children in urban environments live within walking and cycling distance to school , there are a number of real and perceived barriers which contribute to the continued decline in active transport rates. These include perceived danger to children from strangers, inadequate safe crossing points and the presence of heavy traffic . The increasing participation of women in the workforce adds time pressure to family living  and also potentially contributes to declining active transport rates.
In countries such as The Netherlands and Germany, where there is a culture of active transport and compulsory education programs for children around active transport and traffic safety, participation rates are much higher and obesity, diabetes and hypertension rates (in adults) are much lower than the USA or Australia . Government strategies in The Netherlands and Germany to increase active transport and improve safety have seen fatalities decline markedly between 1975 and 2001 and urban environments changed to improve conditions for active transport participants .
As an active transport to school option, the Walking School Bus (WSB) program offers a potentially important strategy to reduce overweight and obesity in children. The purpose of this paper is to assess, within the context of the Assessing Cost-Effectiveness of Obesity (ACE-Obesity) project , the potential cost-effectiveness of the WSB program as an obesity prevention measure for primary school children. The program was not intended as a dedicated obesity prevention measure, and this study acknowledges its multiple benefits. Whilst not endeavouring to quantify them, we investigated the impact of attributing a share of the program costs to non-health objectives.
A cost-effectiveness evaluation was undertaken, and the incremental cost-effectiveness ratio (ICER) calculated as the cost ($AUD) per Body Mass Index (BMI) unit saved and disability-adjusted life year (DALY) saved. The intervention time horizon was one year, which reflects how the program would be applied in real life. It was assumed to be in steady state (i.e. it is implemented and working at its full effectiveness potential and trained personnel or infrastructure are available). The time horizon for measuring the associated health care cost-offsets and DALY benefits was rest of life or 100 years. All costs and benefits were discounted at 3% in accordance with the recommendations of the US Consensus Panel on Cost-Effectiveness . The reference year was 2001. In addition to the technical results, the intervention was assessed against a series of second stage filter criteria ('equity', 'strength of evidence', 'acceptability', 'feasibility', 'sustainability' and 'side-effects'). This analysis involves the assessment of issues that either influence the degree of confidence that can be placed in the cost-effectiveness ratio, or broader issues that need to be taken into account in the decision-making about resource allocation. The economic evaluation methods are detailed in a separate paper (submitted to BMC Public Health 25 March 2009) and in .
The proposed intervention was based on the WSB program as operated under the auspices of VicHealth in Victoria, Australia. The program aims to increase the number of children of primary school age walking to school . Children are accompanied by 2 volunteer adult "conductors" (at a ratio of 1 adult to 8 children) and travel along a set route through a neighbourhood picking up children along the way at designated stops and delivering them to school. Conductors complete an induction program, are registered and insured under their local government's volunteer policy. All volunteers must complete a satisfactory police check .
The comparator for this intervention was 'current practice', defined as 'do nothing' as there was no organised program was in place. Before implementation of the WSB in Victoria, approximately 27% of Preparatory to Grade 2 children (ages 5-7 years) and 24% of Grade 3 to 6 children (8-11 years) walked to school every day .
Assessment of Benefit
The first stage of benefit assessment involved estimating the health gain attributable to the intervention using the DALY. This required calculation of the increase in physical activity due to the intervention, conversion to BMI change as children and then conversion to DALYs and cost-offsets over their lifetime . There are no reported trials of the impact of the WSB interventions on physical activity, weight or BMI. Therefore, a range of other available data was used to model the likely change in the BMI of individual participants of the WSB program who were new to active transport (Table 1). It was assumed that there was no effect of a change from car transport to active transport on physical activity at other times or energy intake levels [10-12].
Table 1. Modelling of reduction in BMI for a single 'average' individual new to active transport (Effects are averaged over one calendar year)
The increased energy expenditure for a child who changed from car transport to WSB was calculated by subtracting the energy costs of walking (3.5 metabolic units [METS]) from the energy costs of sitting in a car (1.0 METS) . The net 2.5 METS was multiplied by the assumed average weight of the target age children (kg) and the assumed time to walk to and from school to derive total increased energy expenditure (kJ/d) (Table 1). The validated method of Swinburn et al. was used to convert changes in energy balance to changes in weight .
Whilst VicHealth had data on the percentage of children walking to school pre-intervention and on current participation of children involved in the Victorian program, there were no data available about the consequent change in active transport rates or the number of participants new to active transport. As anecdotal evidence indicated that some participants previously walked to school with a parent, it was assumed that, of the children participating in the WSB intervention, 50% were new to active transport. However, an uncertainty range of 25-75% was placed around this parameter.
Simulation of the intervention
VicHealth funds local municipalities to implement the WSB program. Local governments, in collaboration with the schools and various State authorities and community organisations, facilitate establishment of the walking routes, registration of volunteers and liability insurance for volunteers. Each local government must engage at least four local primary schools during the project. The schools are responsible for the recruitment of both volunteer conductors and children to the program, and for the day-to-day operation of the program. The funding is dependent on the meeting of minimum standards, including the provision of police checks, insurance and training for volunteers and the safety auditing of routes.
It was assumed that the delivery model reflected the intervention operating in 'steady state' in terms of recruitment, uptake and delivery. The current recruitment experience of the Victorian WSB Program , shown in the left hand column of Figure 1, was extrapolated to a national level (right hand column). However, the modelling assumed that fewer local governments in other states or territories would be involved in the program because a higher proportion of their population was rural or remote, and lower relevance of the program to their population. It was also assumed that enrolled children were confined to the first three years of primary school (Prep to Grade 2), as is typical of most enrolees. The number of volunteer conductors involved with a WSB ranges from one to five. For the purposes of training, four volunteers per bus were allowed for, giving a total of 8,960 volunteers.
Figure 1. Recruitment to the Walking School Bus intervention. a Assumes lower percentage of municipalities involved than under the cost-efficacy trial
The benefits at the individual level in terms of reduced BMI and DALYs saved were then applied to participating children new to active transport (50% × 15,680) from the Australian childhood population (age 5-7 years) .
Assessment of costs
Pathway analysis was used to identify the component activities of the intervention in order to ascertain the associated resource utilisation (Figure 2). The costs included, unit costs and their sources, and the assumptions employed are specified in Additional file 1. All costs were adjusted to real prices in the reference year using the relevant Consumer Price Index .
Figure 2. Intervention pathway.
Additional file 1. Unit costs, data sources and assumptions. Unit costs are provided for all resources used in the delivery of the intervention. Data sources are specified, as well as any assumptions employed. $AUD Australian dollars; EFT equivalent full-time; WSB Walking School Bus
Format: DOC Size: 49KB Download file
This file can be viewed with: Microsoft Word Viewer
Given the societal perspective, costs to participants and their families, and all sectors involved in the delivery of the intervention were included (Figure 2, Additional file 1). Since the intervention was assumed to be operating in 'steady state', costs involved in set-up, research and development prior to the recruitment phase were not included. Other costs excluded were those associated with the implementation by local government of any improvements along proposed bus routes identified as necessary prior to commencement of a WSB and the costs of monitoring and evaluation above more than a routine level.
Uncertainty analysis was used to address issues of uncertainty in the results due to sampling error (e.g. in the WSB 'snapshots') and the need to make assumptions due to the lack of evidence surrounding some parameters. The key parameters included in the uncertainty analysis are listed in Table 2. Simulation-modelling was used to facilitate the presentation of an uncertainty interval (median and 2.5 and 97.5 percentiles) around the health benefits, costs and ICERs. The @RISK software was used to conduct Monte Carlo simulations which allow multiple (4,000 in this instance) recalculations of a spreadsheet, each time choosing a value from the specified distribution for each input variable. The ranges presented can be interpreted as the range within which the true value lies with 95% certainty. In addition, the @RISK analysis allows identification of the model parameters that contribute most to the uncertainty in the results.
Table 2. Uncertainty analysis
Sensitivity testing was used to explore the impact on the ICERs of measures designed to trim costs and enhance recruitment. The following scenarios were modelled as univariate sensitivity tests: attribution of 50% of costs to non-obesity related objectives (such as reducing traffic congestion); annuitisation of fixed costs; improvements in capacity utilisation and recruitment (by increasing the number of children per WSB, WSBs per school and schools per local government, and by increasing the proportions of local governments involved and of participants new to active transport). The impact of combining these measures was tested under both an "optimistic" and a "very optimistic" scenario.
The incremental effect of the WSB intervention was a reduction of 0.03 (95%UI 0.01; 0.11) BMI units per child taking up walking to school (Table 3). This is about double that in Table 1 primarily due to the uncertainty range used around the number of days walked per week.
Table 3. Cost-effectiveness results ($AUD)
The incremental costs associated with the intervention were $22.8 M, which translated to an average cost of around $2,900 per year to get one child walking to and from school one day per week (=$22.8 M/7,840). The majority of costs related to central coordination (at national and state level) and to the recruitment of local governments and schools. The local government project officer time was by far the largest single component of costs ($14.4 M or 63%). Fixed costs constituted $19.5 M or 86% of total costs. Volunteers incurred $1.8 M (7.9%), all of which constituted time costs.
The intervention produced modest cost-offsets arising from future reductions in obesity-related diseases (Table 3). The resultant net ICER of $0.76 M per DALY saved far exceeded the commonly used threshold value of cost-effectiveness in Australia of $50,000 per DALY. All of the iterations fell within Quadrant 2 of the cost-effectiveness plane, in that additional costs were incurred in achieving the health gain arising from the intervention (Figure 3). The probability of the ICERs being less than $50,000 was zero. The ICERs were unacceptably high on base run assumptions (both with and without volunteer costs), and the uncertainty intervals very wide. The main contributor to uncertainty in the ICER (cost per BMI) was the number of days of active transport per week (r = 0.56), which ranged from 0.5 (one way, one day) to 5 (both ways, 5 days) in our modelling, with 1 being the most likely value (Table 2).
Figure 3. Cost-effectiveness plane -- Net cost per disability-adjusted life year (DALY) saved (with cost-offsets).
Sensitivity tests results
Cost-cutting measures including attribution of a portion of the total intervention costs to non-obesity related objectives and the annuitisation of fixed costs alone were insufficient to make the intervention cost-effective (Scenarios 2-4, Table 4). When the improved capacity utilisation measures (Scenarios 5-8) were added to the cost cutting measures (Scenario 11), the intervention was cost-effective, but only under the 'very optimistic" scenario. The inclusion of an increase in the proportion of children receiving the benefit meant the intervention approached cost-effectiveness under the "optimistic" scenario (Scenario 13).
Table 4. Sensitivity testing ($AUD)
Second stage filter analysis
A consideration of 2nd stage filters for the intervention is summarised in Table 5. The key decision points relate to the weak evidence of effectiveness for the intervention, and some significant concerns around its feasibility and sustainability. The program potentially offers some significant wider positive side-effects.
Table 5. Second stage filter analysis
The WSB intervention was not effective or cost-effective in terms of its effect on obesity in children under the current assumptions. The ICERS, even with cost-offsets included, were very large. The reasons for the intervention's poor performance in terms of its cost-effectiveness are several. Firstly, it was modelled on the basis of the relatively low recruitment and uptake of the program in Victoria as at 2004.
It could be reasonably argued that the program has not yet reached steady state. There appears to be considerable unutilised capacity within the current delivery model, and consequent scope for expansion of the program within the current infrastructure arrangements. The number of participating local governments and schools, the number of WSBs per school, and children per bus, could all potentially be increased without expansion of the infrastructure capacity at either the national, state, local government or school level. Sensitivity testing showed that improvements on a number of fronts are required to bring the intervention into the realm of cost-effectiveness as an obesity reduction measure. However, whilst cost-effectiveness could be improved by a more comprehensive coverage and uptake of the program, the number of children required to participate at these levels may not be realistic in the Australian context.
The WSB was not designed as an obesity prevention initiative, despite often being marketed as such http://www.vichealth.vic.gov.au/Programs-and-Projects/Physical-Activity/Active-and-Safe-Travel-to-School/Walking-School-Bus.aspx webcite. The program was intended to produce change in the active transport behaviour of students. It has a number of objectives, all of which may result in potential positive side-effects (such as a safer traffic environment around schools, reduced congestion, accidents and pollution), provided that the intervention is shown to be effective in increasing the number of children new to active transport. Whilst these other benefits were documented in the second stage filter analysis, they were not incorporated into the technical results. It could be argued that the costs associated with the intervention should not have been fully attributed to the BMI outcomes, but be apportioned across a range of program objectives as was done under Scenario 2 of the sensitivity analysis. Given our adoption of a societal perspective, the choice was to capture all benefits and all costs, or alternatively, to capture only benefits and costs relevant to our objective. The latter path was chosen as data were lacking on broader benefits enabling their quantification. Whilst this evaluation concluded that the WSB was not a cost-effective intervention for obesity, its merits and potential benefits in terms of a range of other outcomes are acknowledged . However, given the high cost of the intervention in terms of each child new to walking to school ($2,900) under the base case assumptions, the other benefits would need to be substantial.
Incorporation of the multiple benefits of the program within the technical analysis was neither appropriate nor possible within the context of the ACE-Obesity study. The ACE methodology is premised on the application of consistent evaluation methods to all of the interventions, and the use of the same outcome measures (in this case, BMI and DALYs saved). Cost-effectiveness and cost-utility analyses were employed to facilitate comparison with the other interventions where the same outcomes were measured. The other benefits are, firstly, not relevant to the focus of the current study, and secondly, no measurement has been undertaken within the WSB program to show that such perceived benefits have actually been realised by the program. An economic evaluation of the WSB program as a stand-alone intervention taking into account all its potential benefits would necessitate new empirical data collection around a range of outcomes, and the potential use of different evaluation methodologies (such as cost-benefit analysis where a monetary value is attached to both costs and benefits).
It may be possible to justify programs which have low direct effects such as WSB and changing school canteens  as 'lighthouse' interventions - meaning that they are visible demonstrations of appropriate actions that 'show the way' for broader actions and eventually changes in norms. Indeed, it is difficult to envisage a broad community effort to promote active transport without a program on school travel and healthy eating for children without school canteen changes. The counter argument to this 'lighthouse' approach is that it fosters a disregard for the importance of applying guidelines of efficiency and effectiveness in decision-making. Acceptance of the 'lighthouse' argument would need to be context specific with strong safeguards built-in around ongoing monitoring and evaluation to achieve 'value-for-money'. Our analysis suggests that more cost-effective programs to promote active transport are required and need to be evaluated. This could be achieved through improvements to the WSB or by different programs given the high cost of the WSB program as it is now.
Lack of data on the incremental change in the numbers taking up active transport as a consequence of the intervention was a key limitation of the evaluation. The limited evidence of effectiveness made the strength of evidence a key decision point. Data used in the modelling of this intervention was drawn from a one week snapshot of the VicHealth program . Only 26 of the 33 participating local governments submitted data, and program activity levels may be under-estimated. However, there are no data showing an increase in the number of children walking due to the WSB intervention. Other important decision points relate to the program's feasibility and sustainability without ongoing funding (Table 5). The information gaps identified highlight the need for better evaluations of public health interventions. Whilst modelling as a tool allowed decisions to be based on the best available data, the results for this intervention are not as robust as they would be if a stronger study designs (such as RCTs) had been used and actual outcomes (i.e. BMI) were measured.
The intervention offers some potential wider positive benefits which were not taken into account. For example, the inclusion of taggers (generally older children, who were not formally enrolled in the WSB program, but "tagged" behind the buses), would increase the number of children participating in the intervention on a national basis from 15,680 to around 19,300. For the taggers, a mean reduction of 0.019 BMI units for boys and 0.021 for girls was calculated and applied. This would enhance the intervention's cost-effectiveness as it would increase the total loss in BMI, without any additional associated costs. However, it should be noted there were no data to verify that these children were new to active transport following the introduction of the WSB program. Further work is required to establish whether there is an increase in the number of children walking and in the time spent in walking and whether the other possible benefits of the program are realised.
Furthermore, this still may constitute a conservative estimate of the benefit, as the intervention involves some whole-of-school activities which may have positive spin-offs to both the wider student population as well as to parents and the wider community. Wider reach and lower costs would improve the WSB credentials as a priority for funding. The central importance of school travel in the quest to increase active transport suggests that improvements in the WSB or its variants need to be developed and fully evaluated.
Whilst, under current modelling assumptions, the WSB program is not an effective or cost-effective measure to reduce childhood obesity, its economic credentials would be improved by more comprehensive implementation within current infrastructure arrangements. The attribution of some costs to non-obesity objectives (reduced traffic congestion and air pollution etc.) is justified to emphasise the other possible benefits. The importance of active transport to school suggests that improvements in WSB or its variants need to be developed and fully evaluated.
ACE: Obesity Assessing Cost-Effectiveness of Obesity; BMI: Body Mass Index; DALY: Disability-adjusted life year; EFT: equivalent full-time; ICER: Incremental Cost Effectiveness Ratio; kg: kilograms; kJ: kilojoules; M: Million; MET: Metabolic Equivalent Unit; SE: standard error; USA: United States of America; WSB: Walking School Bus; $AUD: Australian dollars.
The authors declare that they have no competing interests.
MM conducted the modelling and analysis of the intervention and assumed primary responsibility for writing of the journal article. MH project managed the ACE-Obesity project and made substantive input into the drafting of the paper. LG was responsible for the literature review and data collection underpinning the modelling. BS provided expertise in the modelling from a change in physical activity to a change in BMI, and also provided valuable comment on the paper. RC provided technical expertise around the economic evaluation protocol and modelling, and provided comment on the paper. All authors read and approved the final manuscript.
The Assessing Cost-Effectiveness in Obesity (ACE-Obesity) project was funded by the Victorian Government Department of Human Services, Australia. We thank members of the ACE-Obesity Working Group for their input into the project: Michael Ackland (Deputy Chair), Bill Bellew, John Catford, Elizabeth Develin, Helen Egan, Bonnie Field, Tim Gill, John Goss, Robert Hall (Chair), Brian Harrison, Kellie-Ann Jolly, Mark Lawrence, Amanda Lee, Tony McBride, Karen McIntyre, Jan Norton, Anna Peeters, Theo Vos, Melissa Wake, Rowland Watson.
Acknowledgement is made to staff of VicHealth, and in particular, Ms Kellie-Ann Jolly, for their input into this intervention.
The opinions, analysis and conclusions in this article are those of the author/s and are not necessarily endorsed by the Victorian Government Department of Human Services, the Victorian Minister for Health or the Government of Victoria.
Am J Publ Health 2003, 93(9):1509-1516. Publisher Full Text
Haby M, Vos T, Carter R, Moodie M, Markwick A, Magnus A, Tay-Teo KS, Swinburn B: New approaches to assessing the health benefit from obesity interventions in children and adolescents: methodologies and a specific example from the Assessing Cost-Effectiveness in Obesity (ACE-Obesity) project.
Int J Obes 2006, 30:1463-1475. Publisher Full Text
Expert Rev Pharmacoeconomics Outcomes Res 2008, 8(6):593-617. Publisher Full Text
Aust New Zeal J Publ Health 2004, 28(2):167-172. Publisher Full Text
Res Q Exerc Sport 2000, 71(3):240-248. PubMed Abstract
Ainsworth BE, Haskell WL, Whitt MC, Irwin ML, Swartz AM, Strath SJ, O'Brien WL, Bassett DR Jr, Schmitz KH, Emplaincourt PO, Jacobs DR Jr, Leon AS: Compendium of physical activities: an update of activity codes and MET intensities.
Goran MI, Nagy TR, Gower BA, Mazariegos M, Solomons N, Hood V, Johnson R: Influence of sex, seasonality, ethnicity and geographic location on the components of total energy expenditure in young children: implications for energy requirements.