The complexities of using international comparisons to guide NHS reform
The complexities of using international comparisons to guide NHS reform
1 September 2023

Key points
- Many of the challenges facing the NHS are not unique to the UK, creating opportunities to learn from health systems in other countries. But drawing meaningful insights from international comparisons of health system performance requires an appreciation of the limitations and complexities involved.
- Health systems have limited influence over the factors that affect needs for health care. Decisions that shape the social and economic conditions that determine a population’s health and health inequalities are taken outside the health system – but these factors are not fully accounted for in international comparisons.
- How well health systems perform is about more than money. But differences in resources – including funding, health workers and equipment – is still important context for understanding performance.
- There are few shortcuts to a systematic and balanced assessment of health system performance. And, while we now have more and better data to support international comparisons, the openly available measures do not offer a balanced picture of what health care services are expected to deliver, with key gaps on primary care, long-term conditions, mental health, and other areas.
- International comparisons of health system performance can highlight where there are differences between countries, but corelation is not causation. Without comparative information about policies and context to guide priorities for reform, copycat changes could result in failed reforms and wasted effort.
Table 1: Health system objectives
Health system goal | Definition | |
---|---|---|
Health improvement | Improvement of the health of the population, including different parts of the life cycle, morbidity and premature mortality | |
People centredness | Approach to care organised around the comprehensive needs of people rather than individual diseases, and with respect to social preferences | |
Financial protection | Safeguarding people against the financial hardship associated with paying for health services | |
Efficiency of the health system | Making the most of the available resources to deliver health improvement, people centredness and financial protection | |
Equity of the health system | The distribution of health improvement, people centredness and financial protection across the population as a whole |
Summarised from: Papanicolas I, Rajan D, Karanikolos M, Soucat A, Figueras J. Health system performance assessment: a framework for policy analysis. European Observatory on Health Systems and Policies; 2022.
Each of these overall objectives is a domain in itself, with multiple measures needed to understand the many different aspects of performance within them. For example, comparing how well people are protected from the financial risks of ill-health would find that universal health coverage has now been achieved by most high-income countries, with the notable exception of the US. But stopping there would overlook important differences in the range of services covered, the charges paid at point of use and the proportion of households that incur catastrophic costs. Equally, an overall national measure is likely to mask substantial variations in financial protection between demographic groups or in different localities.
While health system performance would ideally be judged on outcomes rather than inputs, activities or processes, outcome metrics (such as reduced mortality or improved quality of life) are often difficult to measure, slow to change and may be affected by a range of other factors. So we also need to look at performance against a range of ‘intermediate objectives’ – essential steps towards achieving the overall objectives that are important in their own right and help us to understand what contributes to better or worse performance. Such measures include the quality of clinical care – measures of access, safety, effectiveness, experience, equity and efficiency at different levels of the health system.
As no health system performs universally well, comparisons of health system performance should look across and within a range of conditions and services. To support this, various frameworks have been developed to promote a systematic and balanced approach to analysing performance – such as the OECD’s HealthCare Quality and Outcomes Indicators and the WHO’s Health System Performance Assessment. While applying these frameworks may be time consuming, especially in looking across multiple countries, excluding important aspects of performance without clear justification risks a skewed and potentially misleading assessment.
While various efforts have been made to combine multiple measures of health system performance into an overall summary measure or league tables, such approaches have significant limitations. Decisions about what to include or exclude in the rating and whether some indicators should be considered as more important than others are inherently subjective, with no consensus on how data should be combined into an overall measure. Even within the same health system, the various ways of calculating an overall rating for individual hospitals may produce very different results. Summary ratings therefore tend to mask the complexity of health system performance and conceal far more than they reveal.
1 in 4 adults and 1 in 10 children experience mental illness, and there are growing concerns about the rising prevalence of mental health problems in the UK. But there are far less international data on mental health than physical health conditions. OECD data on the quality of mental health care is limited to rates of suicide and excess mortality among patients diagnosed with a mental health disorder. The limited scope for comparing mental health care is further constrained by how few countries report these indicators to the OECD, which recently called for ‘bold action’ on mental health in the wake of COVID-19.
Some aspects of health care performance that matter to patients and the public are hard to measure, not measured in all countries or not measured at all. For example, waits for hospital treatment is one of the public’s top priorities for the NHS. While the UK devotes considerable effort to measuring waiting times and lists, only a few other countries do similarly and so the scope for comparisons of this aspect of performance is limited.
This is not intended as a criticism of the organisations working to collect, analyse and continuously improve the data. But it is an important limitation to be aware of when interpreting the available data.
The International Cancer Benchmarking Partnership (ICBP) was created in 2009 to produce research comparing cancer survival, incidence and mortality across high-income countries with comparable registry-based data and identify factors driving differences to improve patient care. The partnership covers 22 jurisdictions across eight countries.
A range of data sources and multiple research disciplines are used to examine areas such as public awareness, the role of primary care in cancer diagnosis, patient pathways and the organisation and structure of health systems. The partnership was initiated by the Department of Health and Social Care in the UK, where survival rates from some cancers are consistently lower than other comparable countries – possibly due to later diagnosis.
Research, including that produced by the ICBP, has examined whether patients in the UK present later than elsewhere, are referred late by GPs or are not promptly diagnosed in secondary care. Despite the impact of the partnership’s early research – and the practical benefits of having more timely, in-depth and policy relevant analysis – the model of the ICBP has not yet been widely replicated for other conditions.
Much of the available data used in international comparisons only offer a glimpse of how well parts of the health system perform for people with a particular condition or who use a specific service. As patients with complex needs may require services from across many settings, being able to compare the quality of the health care delivered across entire patient care pathways is becoming increasingly important. International collaborations such as ICCONIC have advanced the use of linking patient-level data across clinical pathways to provide more in-depth insights than the existing aggregate measures. However, these are constrained by major gaps in the data – limiting comparisons of pathways that span primary, community and social care to what happens in acute settings.
Further reading
Work with us
We look for talented and passionate individuals as everyone at the Health Foundation has an important role to play.
View current vacanciesThe Q community
Q is an initiative connecting people with improvement expertise across the UK.
Find out more