The use of benchmarking and register data to assess the performance of orthopaedic care

Aim of the project

To support organisations in improving performance, benchmarking is a valuable tool for several reasons. First, it is an effective substitute for competition in the public sector. Second, through benchmarking, performance differences between organisations in various measures are  revealed, and for those performing less optimally, possible improvement areas may be discovered. Third, benchmarking is useful for evaluating the impact of healthcare reforms on performance. Coupled with big data from patient registers and other administrative registers, benchmarking thus can offer opportunities for finding ideal structures in the provision and financing of healthcare. The overall aim of this dissertation project was to show how benchmarking can be applied to assess healthcare performance with the use of register data.

Methods

The four studies included in the project were based on two comprehensive patient-level datasets, with data obtained from multiple registers. Study I applied international benchmarking, with the performance for the surgical treatment of hip fractures being assessed between and within seven European countries. Regression analyses were used to explore associations between age- and sex-adjusted mortality rates and length of stay (LOS) and selected country- and region-level variables.

In Studies II–IV, a national perspective was considered in the assessment of the performance of elective hip replacement surgery in Sweden. In Study II, the orthopaedic departments’ productivity development between 2005 and 2012 was measured by the Malmquist Productivity Indices. The indices were further decomposed into changes in efficiency and technology. In Studies III and IV, a quasi-experimental research design was applied to assess the effects of a healthcare reform involving competition and financial incentives introduced in the capital region in 2009. In both studies, difference-in-difference analysis was used to estimate the causal effects on LOS and various measures of subjective and objective quality. In Study III, the difference-in-difference analyses were also stratified by hospital type to examine whether the reform had heterogeneous effects across hospital types. In Study IV, an entropy balancing algorithm was further applied to make the intervention and control groups comparable.

Results in brief

Study I revealed marked differences in age- and sex-adjusted LOS and mortality rates for hip fracture patients, across and within included countries. Variations were found to be associated with the availability of national clinical guidelines, the share of males in the region and country-specific effects.

In Study II, differences in the development of productivity, efficiency and technology in the provision of hip replacement surgery across and within the orthopaedic departments were revealed. The overall results indicated a slight positive productivity development over the study period, which was primarily due to catch-up effects (improvements in efficiency), rather than changes in technology.

The findings from Study III indicated that the reform led to the LOS of the surgical admission not decreasing at the same rate as before, and to reduction of the adverse event rate within 90 days following surgery. These effects were driven mainly by university and central hospitals. Furthermore, the reform brought no changes in patient satisfaction with the outcome of the surgery (Study III and Study IV) or gains in various patient-reported outcome measures at one- and six-year follow-ups (Study IV).

Conclusions

The PhD-project has demonstrated how benchmarking can be applied to assess healthcare performance with the use of register data, with the four studies contributing with various perspectives and measurements at different levels of healthcare systems. First, the thesis has exemplified how performance measurement can be applied to identify and analyse performance gaps. Considerable variations in the performance of orthopaedic care between and within units of analysis were revealed at the departmental and international levels. This implies that there is room for improvement and that stakeholders should learn from best practices.

Second, the thesis has demonstrated how benchmarking can be useful in the assessment of healthcare reforms. The findings indicated that the studied reform reduced the adverse event rate, led to LOS not decreasing at the same rate as before and had no effect on patient- reported outcome measures. These findings contribute to the general knowledge about the effects of market elements and financial incentives and can be used to inform decision making. Future perspectives should focus on how this information can and should be used in practice to change organisations’ behaviour and to improve healthcare performance.

Dissertation and publications

Goude F. Hips don't lie - the use of benchmarking and register data to assess the performance of orthopaedic care. Karolinska Institutet, 2021, supervisor: Clas Rehnberg.

Medin, E., Goude, F., Melberg, H.O., Tediosi, F., Belicza, E., Peltola, M., EuroHOPE Study Group. European regional differences in all-cause mortality and length of stay for patients with hip fracture. Health Economics. 2015;24(Suppl 2):53-64.

Goude, F., Garellick, G., Kittelsen, S.A.C., Nemes, S., Rehnberg, C. The productivity development of total hip arthroplasty in Sweden: a multiple registry-based longitudinal study using the Malmquist Productivity Index. BMJ Open. 2019;9(9):e028722.

Goude, F., Kittelsen, S.A.C., Malchau, H., Mohaddes, M., Rehnberg, C. The effects of competition and bundled payment on patient reported outcome measures after hip replacement surgery. BMC Health Services Research. 2021;21:387.

Contact person