Health system performance across the UK: why we need more comparable data

The UK is a patchwork of kingdoms and since its inception the NHS has been administered differently in each one. Since devolution, policy differences have become much larger. But how can we keep track of what’s working and what’s not, when each country collects data differently?

We talk to Nicholas Mays, Professor of Health Policy at the London School of Hygiene and Tropical Medicine, about the challenges of sourcing comparable data his team encountered when compiling The four health systems of the United Kingdom: how do they compare?, new research recently published by the Health Foundation and Nuffield Trust.

Tell us about the research

This is the fourth in a series of analyses comparing performance of the UK health services. The first was undertaken before political devolution and since then we’ve tracked progress three times. We want to understand whether performance of countries has converged, diverged or changed in relative terms over time. To do that, we selected a set of indicators that were available across the four countries.

The problem is that even before devolution, the number of indicators available to measure inputs, processes and outcomes on a comparable basis was pretty limited. We’ve been trying to replicate that initial data set, and in some cases we’ve been able to bring in additional sources of information, but basically we’ve been struggling with comparability of data for each analysis.

Which data sets were most problematic?

Well, we’ve always been able to compare mortality rates, due to the way deaths are certified, and indeed the fact that death itself is fairly unequivocal! So we were able to look at life expectancy and also what’s known as ‘amenable mortality’, in other words deaths that, with reasonable modern health care, shouldn’t happen, or should happen very rarely.

But with almost everything else, we encountered problems of changing definitions over time between the four countries. For example, day patient rates are a good measure of efficient use of hospital resources, but it’s virtually impossible to compare them over time because categorisation has changed in different countries. In relation to staffing, it’s very difficult to get data for GPs across the whole of the UK from before devolution to the present.

Since devolution, the availability of comparable data has actually declined. So it’s harder, for example, to compare expenditure data across the countries. You can’t really pull out, say, spending on hospitals from GP spending, in the way that you need to. It’s got harder to compare nurse staffing levels because the way that trainees, agency staff and so on are counted has diverged between the countries. To really build a picture of how the health systems are performing in relation to one another, you need access to more readily available comparative data between all four countries, which we simply don’t have at present.

What were the key findings of your research?

On the whole, health care performance across the UK has been improving. In terms of population health and amenable mortality (the results of health care), each of the countries seems to have made similar progress since 1990, and rates of improvement are roughly similar across the four countries. The relative positions of the four countries have not altered.

If you regard amenable mortality as a reasonable measure of health service performance, then it would suggest that whatever the policy differences, up until now, they haven’t been enough to produce divergence in performance between the countries.

So you could conclude that the impact of devolving power to run the health service to Scotland, Wales and Northern Ireland has not been as dramatic as you might have expected. One interpretation of these findings is that the more important influences on health system performance are how we apply innovations in drugs and technologies, and produce high quality staff. Although politicians get excited by differences in health care organisation – such as whether providers compete, how services are commissioned and how services are configured – ultimately these other developments may be more important.

Why does this research matter?

One reason we started this work was because we knew that since the 1970s at least, Scotland and Northern Ireland – and to a lesser degree, Wales – had had higher levels of health care funding per capita than England. Of course post-devolution the policy differences have also become much larger. So we were originally interested in whether additional expenditure was leading to better results and better quality care, and after devolution whether the different system policies were having a visible impact as well.

There’s also the more analytical learning opportunity of having four of the most similar health systems in the world available for scrutiny. Cross-country comparisons are useful, but the problem you normally face is that the objectives of systems and underlying values are rather different. But in this case, as far as we can see, the four UK health systems are still broadly pursuing the same goals, yet they’ve made different policy decisions around things like user charges and organisation, and about how much to spend on health care versus social care and other services. This makes for interesting comparisons.

What do you see to be the priorities for future research and data collection?

We feel it’s time to do more detailed studies of particular services and client groups (eg people with diabetes or after a stroke), maybe by having matched groups of patients going through the health system on either side of the England-Scotland/Wales borders at the same time. We’d also like to compare performance at district or regional level within and across the four countries.

In the report we also recommend that there should be a focus on collecting core, like-for-like information across the four countries about inputs (eg expenditure and staffing), processes and activity, and a range of data on quality and outcomes. One thing we’d like to see happen – which we know is politically sensitive – is for more of the National Clinical Audits to become genuinely UK-wide, assuming of course that the UK stays intact. These tend to include England plus Wales and usually Northern Ireland, but almost never Scotland. But overall we feel that there really does need to be more comparative data made available across the four countries, so that we can effectively benchmark and learn from previous experience.