Evaluation of Warm Up New Zealand: Heat Smart programme

    Background

    The Warm Up New Zealand: Heat Smart (WUNZ:HS) programme began on July the 1st 2009. WUNZ:HS is a $340 million dollar multi-year programme funded by the New Zealand government that provides funding for insulation retrofits and clean, efficient heating grants for New Zealand households. It is the largest programme of its type in New Zealand history and represents, amongst other things, the fruition of more than a decade of academic and applied research by He Kāinga Oranga, the Housing and Health Research Programme of Otago University, Wellington: a strong example of public health research demonstrably guiding and influencing public policy.

    Although WUNZ:HS is administered by the Energy Efficiency and Conservation Authority (EECA), an organisation whose core mandate is improving the way that New Zealand uses energy, the health benefits of improved insulation and heating, which have been demonstrated by work done by He Kāinga Oranga amongst others, were clearly factored into the justification of the WUNZ:HS programme.

    In 2010 the Ministry of Economic Development (MED), EECA's parent organisation, tendered a contract to carry out a full cost benefit analysis of the programme. The bid was won by a consortium including academics from He Kāinga Oranga and Victoria University, and consultancy firms Motu Economic and Public Policy Research and Covec.

    He Kainga Oranga members Professor Philippa Howden-Chapman, Dr. Lucy Telfar-Barnard and Nick Preval worked with Dr Richard Arnold of Victoria University on the analysis of health related data which made up an element of the cost benefit analysis.

    Aim

    To evaluate the benefits and costs associated with the WUNZ:HS programme, and produce an estimate of net benefit.

    Design

    Strongly informed by Dr. Lucy Telfar-Barnard’s PhD thesis, the study pairs homes that received a retrofit under the WUNZ:HS programme with control homes that are similar in age, size, quality and location via anonymised matching by a third party organisation. Anonymised data is collected from the Ministry of Health for the people who live at these homes including health outcomes and demographic information. Anonymised energy use data for these addresses is also collected from energy companies.

    Analysis of the relationship between receiving a retrofit and health or energy use outcomes is then possible at both the individual level and household level.

    From the initial list of homes that received treatment under the programme between June 2009 and May 2010 (46,655), the final usable data set included 255,672 treatment and control households and 973,710 individuals.

    In addition a separate assessment of the impact of the programme on employment was carried out.

    Main Outcome Measures

    • Ratio of benefits to costs for programme as a whole
    • Measurement of programme costs
    • Changes in employment due to programme
    • Changes in electricity and gas use due to programme
    • Changes in hospitalisation rates and mortality rates (He Kāinga Oranga)
    • Changes in hospitalisation and pharmaceutical costs per household (He Kāinga Oranga)

    Results

    Our health analysis included both individual and household level data. Individual level analysis utilised negative binomial models to analyse the impact of participating in WUNZ:HS on hospitalisation rates and mortality rates for recently hospitalised older people. We found that there was no statistically significant change in hospitalisation rates as a result of participating in WUNZ:HS but that there was a statistically significant 27% reduction in mortality for participants aged 65 and over who had recently undergone a cardiovascular hospitalisation. We estimated that this on-going benefit could be valued at $439.95 per year per treated household.

    At a household level we utilised a fixed effects OLS estimator with standard errors clustered by treatment/matched control pairings to analyse hospitalisation and pharmaceutical use costs. We found that there was a statistically significant saving of approximately $64.44 in total hospitalisation costs per year for a household that received some combination of ceiling or floor insulation under the WUNZ:HS programme; a $67.44 yearly saving in circulatory illness related hospitalisation costs, a $98.88 reduction in respiratory illness related hospitalisation costs and for asthma-related hospitalisation costs (a subset of respiratory illness) a higher saving at $107.52. We speculated that this seeming discrepancy was likely the result of statistical noise. Sub-analyses by Community Services Card holders status (of treated homes) suggested that the cost reduction resulting from insulation retrofits was gained primarily by households that participated in WUNZ:HS as Community Services Card holders. Pharmaceutical savings were small but highly statistically significant for insulation, and not statistically significant for heating.

    We speculated that the difference between household level costs and individual level hospitalisation rates could be explained by the duration of stay and cost of procedures that are factored into hospitalisation costs but not events data. It was for this reason that our household level results informed the final cost benefits analysis.
    Motu primarily used a fixed effects OLS estimator with standard errors clustered by treatment/matched control pairings to analyse changes in total energy use and electricity use as a result of receiving an insulation or heat pump retrofit under WUNZ:HS.

    Their favoured model found that there was a 0.96% reduction in average annual household electricity use as a result of receiving an insulation retrofit under the programme and 0.66% reduction in annual total metered energy used. Other key findings included a 1.92% increase in electricity use as a result of heat pump installation and a 0.75% increase in total metered energy used.

    When health and energy results were combined with an analysis of industry impacts and employment changes a final cost benefit analysis was carried out. The results of the cost benefit analysis were highly favourable. Under the preferred scenario, (additionality assumption of 85%, 4% discount rate) it was estimated that WUNZ:HS will have a net benefit of $951 million dollars, and a highly favourable benefit cost ratio of 3.9:1.

    Reports 

    Back to top