Implementation Process: Appendix CC

Joint Crash Reduction Programme: Outcome Monitoring

Abstract

Introduction

The Land Transport Safety Authority (LTSA), Transit New Zealand and Local Authorities are partners in the Joint Crash Investigation Programme in New Zealand, which was set up in 1985. This is a continuous programme of systematic investigation of all roads in New Zealand. The programme identifies suitable sites for treatment based on the crash history at each site, and recommends low cost engineering treatments aimed at reducing those crashes.

Since 1989 the then Land Transport Division of the Ministry of Transport (now the Land Transport Safety Authority) has progressively developed a monitoring system to gather data on sites investigated in this way, in order to evaluate the outcome of the programme. Regular evaluations of the programme and of specific interventions have been carried out (Kraus, 1991; Land Transport Safety Authority, 1995-2001). This monitoring system has now been redeveloped to be compatible with the LTSA's Crash Analysis System (CAS). CAS uses precise geocoding of crashes and crash reduction monitoring sites to generate more accurate spatial analysis of data than was possible under the earlier Accident Investigation System (AIS). The monitoring system on CAS also incorporates a revised method of controlling for crash reductions across the network.

Redevelopment of the monitoring system and control mechanism was completed early in 2001. This paper discusses data sources and techniques used to evaluate the programme, and presents the results of two analyses carried out using the CAS-based system: a recent overall evaluation and an evaluation of one specific intervention (installation of throat and fishtail islands at intersections).

Data sources

Monitoring site data

Details of the crash problems identified during the investigation and the actions recommended to address these are recorded in the Crash Reduction Study Monitoring System database, which now forms part of the LTSA's Crash Analysis System. In addition to the study recommendations, this database includes a range of information about the investigated sites including layout, engineering features and traffic flow. Each site is also tagged with the date at which post-implementation monitoring is to stop. Typically this is five years from the end of works, but may be less for some types of intervention, or where the action taken as part of the crash reduction programme has been superseded by other works. Only crashes occurring within each site's monitoring period were used to calculate crash reductions.

The LTSA's Crash Analysis System (CAS) contains details of fatal and injury motor vehicle crashes reported to the New Zealand Police (Frith, 2000). In recent years non-injury crashes have been included in the database and may be used in identifying sites and appropriate crash reduction treatments. As reporting rates for these are lower and more variable than for injury crashes, evaluations of the programme to date have been based on an analysis of injury crashes only.

For each reported injury crash, a Traffic Crash Report with details of the crash including time, location, road conditions, a description of the crash and a diagram showing crash site layout and key movements in the crash, is completed by Police. This information is entered into the Crash Analysis System, and each crash is coded to indicate its exact location. This enables easy identification of crashes which occurred at monitoring sites. By combining site and crash information from the database in this way it is possible to calculate crash reductions for a range of site and crash types.

Methodology

Controls

Need for a control mechanism

A simple comparison of injury crash rates before and after treatment at a site is likely to reflect changes which are related to changes in the wider road safety environment as well as those due to the low-cost site treatment. Factors such as changes in traffic volume, drink-driving prevention programmes, changes in vehicle speeds (whether resulting from speed enforcement or from increased traffic congestion) and so forth may affect national or regional crash rates including, of course, those at the study sites.

In order to control for this, the underlying crash trends may be assessed by looking at a comparison group of crashes occurring outside treated sites. This comparison group is chosen to be as similar as possible (in terms of factors affecting crashes) as the sites being studied, so that the major difference between them can be ascribed to the site treatment(s) applied.

Identifying suitable comparison groups

Prior to 2000, the comparison group for sites within a local body and urban or rural speed limit area was defined in terms of similar levels of traffic growth (defined as high, medium or low). This method addressed a major factor underlying variations in crash patterns. However, traffic growth itself changes over time, so that an area which previously experienced low traffic growth may undergo a boom in housing, employment or recreational opportunities leading to much higher traffic growth. Under this method, when a site or group of sites was moved from one comparison group to another, (e.g., from the 'low growth ' to 'medium growth' groups) the estimated crash reductions changed abruptly. This artefact of the control method made it difficult to estimate consistent results for an area or type of treatment over time.

The current method uses geographical comparison groups. Each site has been assigned a comparison group consisting of injury crashes before and after implementation in a comparable geographical area. A compromise had to be found between using too large an area, which might artificially group places with very different growth patterns or other factors affecting crashes, and using too small an area which would make estimates vulnerable to small random fluctuations in crash numbers. Where possible, the comparison group chosen was roads of the same broad speed limit category (urban or rural) within each Local Authority. If crash numbers in the Local Authority were too small to give a reasonable estimate of the underlying trends, the control area was extended to include roads within the Local Government Region. Only crashes occurring outside designated monitoring sites were included in the comparison group.

Some adaptations to the above scheme were necessary. Christchurch and the remainder of Canterbury were treated as separate "regions" for the purpose of assigning comparison groups. Two regions, Gisborne and the West Coast, had too few crashes to give a reliable estimate of crash trends. Crashes in Gisborne were added to those in Hawkes Bay to give a comparison group for Gisborne Region sites. There were enough rural crashes in the West Coast Region to serve as a comparison group for crashes at rural sites in the region, but for urban sites it was necessary to add West Coast crashes to those in non-metropolitan Canterbury. Lastly, although Franklin District forms part of the Auckland Local Government Region, its traffic patterns are more similar to those of the Waikato Region. Accordingly, the comparison group for sites in the Franklin District was made up of crashes in the Franklin District plus the Waikato Region.

The number of injury crashes at each site in the period before improvement was adjusted for underlying crash trends in the local area, to give an estimated number of injury crashes expected if the improvements had no effect. Comparing this number with the actual number of injury crashes after improving the site gives the crash reduction result.

The expected number of injury crashes at a site or group of sites was calculated as follows

where

CrashesExpected is the expected number of injury crashes at the site in the 'after' period (ie the period of monitoring after all treatments were implemented), assuming the treatment had no effect;

BeforeCrashes is the actual number of injury crashes at the site in the (usually five-year) period before treatment;

ControlBefore and ControlAfter are the actual number of injury crashes in the control area during the site's 'before' and 'after' periods respectively.

Actual and expected numbers of 'after' injury crashes were summed across the chosen group of sites and the totals compared to give the crash reduction result as

Crash reduction studies are potentially subject to regression to the mean bias. Sites for crash reduction monitoring are selected on the basis of a high number of crashes over a given period, usually five years. This carries with it a risk that some sites with a low average rate may meet the selection criteria, due to "highs" in the random fluctuations over the given period. It would be expected that the crash rate at such a site would be lower in the period following selection, even in the absence of any improvements, reflecting the true underlying crash rate. The benefits of works at such sites would therefore be overestimated, as some of the apparent reduction attributed to the site treatment would be due merely to the expected regression to the mean (Hauer, 1997).

The shorter the selection period, the greater the potential bias due to random fluctuations. The selection period used in LTSA crash reduction monitoring is typically five years, which may provide some buffer against single large "blips". No further corrections for possible regression to the mean have been applied. Methods for doing this are under investigation. When regression to the mean is taken into account, crash reductions attributable to the programme may be smaller than the changes quoted here.

Results of the overall programme evaluation

The most recent evaluation of the effect of the crash reduction monitoring programme was carried out in February 2001, using the method described above to control for underlying trends in crashes. Injury crash data to the end of June 2000 were available for this evaluation. Although the low-cost actions recommended are generally aimed at reducing specific types of crashes at a site, an overall reduction in all crashes can be expected as a result of the programme.

As at February 2001, there were 3919 sites in the monitoring database, with works completed at 62% of these. The estimated overall reduction in injury crashes at these fully treated sites was 29%, a total saving in social cost of approximately $2.4 billion since the programme began. (These figures do not take regression to the mean into account). Open road sites (39%) reduction and intersection and non-intersection sites (40% and 58% respectively) experienced the greatest reduction in injury crashes following treatment.

Fig 1: Injury crash reduction by site type and road type

Results of a selected treatment: installation of throat or fishtail islands at intersections

Evaluations of selected treatments are carried out from time to time. An evaluation of the effect of installing throat or fishtail islands at intersections was carried out in 2001 using injury crash data to the end of December 2000. Throat and fishtail islands are grouped together for crash reduction monitoring purposes, and cannot be separated in this analysis.

134 intersections where throat or fishtail islands had been installed were analysed. The majority of these sites were in urban areas on local roads, and most were X-intersections. Although it was not possible to separate the effect of the installation of throat and fishtail islands from other works carried out at the treated intersections, in most cases (99 of the 134 selected sites) the installation of islands was the major component of the site treatment and would be expected to be the main contributor to the crash reduction. At the remaining 35 sites examined here, throat or fishtail islands were installed with other site treatments but were expected to have some contribution to the safety benefits.

Overall, injury crashes at sites treated with throat or fishtail islands decreased by 44% following treatment, an estimated social cost saving of approximately $95 million. If analysis was restricted to the 99 sites where the installation of islands was a major component of site treatment, a 45% reduction in crashes was found. More than half of the total reduction in injury crashes came from crashes involving vehicles crossing paths at the intersection. Crashes of this type decreased by nearly 60% after treatment, from 57 injury crashes per year (after adjusting for underlying trends) to 25 per year (see figure 2 below).

Fig 2: Injury crash reductions by crash movement type

Substantial crash reductions were observed for fatal, serious and minor injury crashes, and for both daytime and nighttime crashes. It is possible that installing islands in the roadway might result in an increase in collisions with obstructions, (ie, the islands); there was no evidence that this was the case, though there is likely to be significant underreporting of such crashes.

Conclusion

Crash reduction monitoring evaluations have resumed using the CAS-based monitoring system. The control mechanism has been revised to overcome difficulties with producing consistent estimates over time which arose with the earlier method. Recent evaluations of the overall programme, and of one specific intervention, have been conducted using the new method, and indicate an overall reduction in injury crashes at treated sites (relative to elsewhere) of 29%.

This paper describes crash investigation outcome monitoring as it is done at present. The process is under continuous review and improvements will be communicated as they occur. Issues for further investigation include ways of dealing with the regression to the mean effect and techniques for assessing reductions in crash severity. Reports scheduled for the near future include evaluations of the effectiveness of installing guard rails, painting/ marking edgelines, and moving/ installing limit lines.

References

Frith, W. J. (2000). Matching crash data with traffic flows and other data related to roads. Presented at the International seminar on road traffic and accident data needs for the new century, Vienna, 25-26 September 2000.

Hauer, E. (1997). Observational before-after studies in road safety. Elsevier Science Ltd, Oxford.

Kraus, C. (1991). Monitoring the effect of the accident investigation programme. Thesis (M.E.) University of Canterbury, Christchurch.

Land Transport Safety Authority, (1995-2001). Accident Investigation Monitoring System evaluation report series. Land Transport Safety Authority, Wellington.