Understanding excess mortality What is the fairest way to compare COVID-19 deaths internationally?

6 May 2020

Holly Krelle
Claudia Barclay

Key points

  • International comparisons of coronavirus (COVID-19) deaths can be misleading, partly because how COVID-19 deaths are recorded differs across countries.
  • Excess deaths is a better measure than COVID-19 deaths of the pandemic’s total mortality. It measures the additional deaths in a given time period compared to the number usually expected, and does not depend on how COVID-19 deaths are recorded.
  • We can compare the impact of COVID-19 between countries by looking at how many more deaths have occurred than we would expect, based on historic trends. In England and Wales in the week ending 17 April 2020 – the peak week – there were 113% more deaths than usual, the same figure as in Spain at the peak of the outbreak there. So far, the peak week figures for France, Sweden and Germany are much lower, ranging from 7% to 53%.
  • In England and Wales, in the period up to 24 April 2020, 71% of the excess deaths were associated with COVID-19. In France the figure for the most recent available week was 92%. This gap suggests that France has been more comprehensive in recording COVID-19 deaths. Some of the excess deaths identified will not be directly due to COVID-19, but may be caused indirectly through pressure on the health care system, or people not accessing care when they need it.
  • Our analysis is based on data from national statistical agencies. EuroMOMO – a source frequently used by commentators – uses different data and a metric based on the increase in deaths relative to normal weekly fluctuations. This metric is not as good for cross-country comparisons and is less intuitively understandable.
  • We have not reached the end of the crisis and data reporting across countries is at different stages. The data available helps us to understand the scale and spread of the outbreak so far, but it is too soon to conclude how the ultimate impact across countries compares.
  • Note: On 19 May 2020 we published an updated version of figure 2. View the updated chart

Introduction

Charts comparing COVID-19 deaths across countries are appearing daily in our newsfeeds. Done well, these international comparisons can help us to understand how different national strategies and policies have affected the spread and severity of COVID-19 outbreaks. But sometimes what is presented in these neat charts is not quite as straightforward as it seems, and can draw misleading conclusions. Here, we summarise some of the ways of comparing countries to help commentators make sense of data on deaths.

Understanding data on deaths

A simple place to start is to look at mortality in different countries via a pair of metrics – the total number of deaths and the deaths per million population (see Table 1). Each of these metrics gives a quite different picture of the countries performing ‘well’ or ‘poorly’. The USA, for example, moves from being ranked number one in total number of deaths, to number nine when adjusted per capita (with the caveat that US deaths do not yet seem to have stabilised). Belgium makes the reverse move, going from number six in overall deaths to being the country with the most deaths per million population. On this measure, the UK – following the recent inclusion of care home deaths – now has the third highest number of deaths in the world.

Table 1: Comparing highest ranked countries by total COVID-19 deaths and by deaths per million

Rank Country Total deaths   Rank Country Deaths per million
1. USA 68,934   1. Belgium 684
2. Italy 29,079   2. Spain 544
3. UK 28,734   3. Italy 481
4. Spain 25,428   4. UK 423
5. France 25,201   5. France 386
6. Belgium 7,924   6. Netherlands 297
7. Brazil 7,321   7. Sweden 274
8. Germany 6,831   8. Ireland 267
9. Iran 6,277   9. USA 208
10. Netherlands 5,082   10. Switzerland 171

 

Source: Our world in data, 5 May 2020 morning update (note that countries of fewer than 100,000 population have been excluded)

Excess deaths: a better comparator?

While these metrics are intuitive and simple, using them as comparators is problematic for the reasons discussed shortly. Increasingly, therefore, commentators are turning to excess deaths to assess different countries’ COVID-19 mortality. This is typically used to count the deaths caused by an event – such as a very cold winter or a pandemic – that would not have occurred if the event had not happened. In this case, we use a measure that compares a country’s weekly deaths in 2020, with weekly deaths in previous years. In England and Wales (the only parts of the UK the ONS reports for), the ONS compares 2020 weekly deaths for all causes to the average of the same week in the previous 5 years.

Figure 1: England and Wales excess weekly deaths (ONS)

Excess deaths is a better measure of the impact of COVID-19 on mortality, and a fairer way to make international comparisons. There are three reasons for this.

  1. The total number of deaths reported across all causes (and therefore excess deaths) is not skewed by differences in defining and reporting COVID-19 deaths across countries. Some countries, including Italy, are only reporting COVID-19 deaths within hospitals while others, including France, are reporting deaths more widely from the community. Definitions also vary, with the UK broadly defining anyone who died with a positive COVID-19 diagnosis as having died because of it. Conversely, in Germany, even if an individual has COVID-19 when they die their death may not be attributed to it.
  2. Measuring excess deaths allows us to understand the full impact of COVID-19 on mortality, including deaths indirectly related to COVID-19. For instance, in Belgium, on 2 days in March, 400 excess deaths were reported not directly attributable to COVID-19. These deaths may be linked to delays in seeking care for other conditions or lower capacity in the Belgian health care system  to deal with other conditions.
  3. Excess deaths enables more valid international comparisons. It allows us to compare a country’s present with their own past, lessening concerns about different population age structures, reporting approaches, or health system capacity. A good comparison measure is the percentage increase of deaths in 2020, over deaths in previous years. EuroMOMO – a source frequently used by commentators – analyses excess deaths using z-scores. This is a measure of the increase relative to normal weekly fluctuations, rather than absolute change – but this is not as good for cross-country comparisons and is less intuitively understandable.

Will the UK have the worst excess mortality in Europe?

Looking at excess deaths data increases the apparent COVID-19 death toll for all countries, including England and Wales. In England and Wales, it shows deaths from the week commencing 21 March as between 10% (in the early stages of the outbreak) and 113% higher (at the peak) than the average of the previous 5 years.  

In addition, if we look at the percentage increase in mortality (Figure 2), not only is the England and Wales peak the joint highest with Spain, the increase remains high while in other countries it has fallen significantly. Recent commentators have suggested that the UK is heading for much higher excess deaths than other European nations – and will trail only the USA in terms of the impact of COVID-19. This is not yet a fair conclusion.

This is because different countries’ data is at different stages of completeness. The ONS, which stands out among national statistical agencies for the quality, frequency and timeliness of its reporting, has data to 24 April. Data from France’s statistical agency is only up to 20 April although corrections are expected for the most recent week. In Italy, it is only for a proportion of the municipalities up to the end of March and then for a smaller proportion of municipalities up to 15 April. Incomplete data is shown by dotted lines in the Figure 2 below. The apparent declines in these countries may turn out not to be ‘real’ – particularly in France and Sweden.

Figure 2: Changes by baseline weekly deaths for selected European countries, by week

Note: On 19 May 2020 we published an updated version of figure 2. View the updated chart

  • England and Wales – Baseline calculated as an average from 2015 to 2019. Data is provisional but large changes are not expected.
  • France – Baseline calculated as an average from 2018 to 2019. Data from the most recent week (10–17 April) may be incomplete based on provincial reporting lags.
  • Italy – Baseline calculated as an average from 2015 to 2019. Data up to the first of April represents 87% of the total municipalities in Italy. Data from April represents only 56% of the municipalities.
  • Spain – Baseline calculated as an average from 2018 to 2019. 2018 data only available from the 9 April that year. Data from the last 2 weeks (18 April–1 May) may be incomplete due to reporting lags.
  • Sweden – Baseline calculated as an average from 2018 to 2019. Data from the last 2 weeks (18 April–1 May) may be incomplete due to reporting lags.
  • Germany – Baseline calculated as an average from 2016 to 2019. Data is provisional but large changes are not expected.

Applications and limitations of the excess deaths metric

The excess deaths measure can be used in several ways to shed light on different aspects of performance.

First, we use it to estimate the number of excess deaths in a given country, rather than simply the percentage increase (see Table 2). How many extra deaths does the UK’s curve suggest? Do we have more excess deaths than Italy? What does this tell us about the absolute scale, pace and impact of the outbreak? While this is easy to calculate from the weekly excess deaths figures (for the UK the figure is around 38,000), comparisons between countries, at this stage, are very tricky. This is because mortality data from countries is at different stages of being reported – as described above. Accurate comparisons of excess deaths will need to wait.

Second, we can look at the proportion of excess deaths attributable specifically to COVID-19. This partly gives us a sense of how comprehensive an individual country’s counting of COVID-19 mortality is. Of the 21,261 excess deaths in France, 92% were recorded as COVID-19 deaths. In England and Wales only 71% were being recorded as COVID-19 deaths. Germany, despite concerns that their tight definition of deaths attributable to COVID-19 was underestimating their mortality, is actually accounting for a similar proportion of their excess deaths as other European nations. In time, we should also begin to be able to unpick what is causing the unaccounted for deaths in different countries – whether these are ‘missed’ COVID-19 deaths, or if these are indirect deaths caused by overloaded health care systems, or delays in patients seeking care for other conditions.

Table 2: Excess mortality in selected European countries

  Excess deaths COVID-19 deaths % COVID-19 Unaccounted for deaths Time period of excess deaths*
England & Wales 38,499 27,222 71% 11,277 21 March–24 April (35 days)
France 21,261 19,627 92% 1,634 16 March–20 April (36 days)
Italy** 32,568 - - - 5 March–15 April (42 days)
Spain 32,466 25,259 78% 7,207 10 March–4 May (56 days)
Germany 1,656 1,216 73% 440 25 March–5 April (12 days)
Sweden 2,469 2,001 81% 468 23 March–24 April (33 days)

 

Source: Excess deaths from national statistics agencies. COVID-19 deaths taken from Our world in data, 5 May 2020 (except for England and Wales data which comes from ONS).

*The start date is when we start counting excess mortality. We start when 2020 deaths are at least 5% above baseline. End date is the latest available data. COVID-19 deaths are also counted within this timeframe.

** Excess deaths are from a subset of municipalities reporting; this cannot be compared to  COVID-19 deaths which are for whole of the country

Making fair international comparisons

By any measure, the UK is one of the countries most affected by COVID-19 in the world. The UK has the third highest total number of COVID-19 deaths and fourth highest number of deaths per million population in the world. Looking at excess mortality – which helps us to include COVID-19 deaths that may not have occurred in hospitals – suggests we are doing worse than Italy, France and Germany, and comparably to Spain. But we have not reached the end of the crisis and data reporting across countries is at different stages. It is too soon to conclude what the ultimate totals will be.

International comparisons will continue to become more robust and insightful as our data improves. Analysis should therefore be updated – and new countries added – as data becomes available.

In making these international comparisons, researchers and journalists should bear two things in mind.

  1. Excess deaths is the best way to compare COVID-19’s impact between countries. When using this metric percentage change above baseline, rather than z-scores, gives a more meaningful and understandable comparison. Be aware that different countries use different baselines, that more recent data is likely to be updated, and that countries are at different stages of their outbreak.
  2. If comparing COVID-19 mortality rates between different countries, be aware that (a) some countries count only hospital-based deaths (b) some countries count both COVID-19 confirmed and suspected deaths, and (c) testing regimes differ between countries.

Further reading

Collection

COVID-19 chart series

The coronavirus (COVID-19) pandemic has delivered a profound shock to the UK. We'll be exploring...

Chart

COVID-19 policy tracker

A timeline of national policy and health system responses to COVID-19 in England.

You might also like...

Chart

Do all care home residents face an equal risk of dying from COVID-19?

Care homes have been on the front line of the COVID-19 outbreak, but why have some areas of the...

Press release

Government must now avoid political distraction and protect social care as a priority

Health Foundation response to ONS weekly deaths in England and Wales.

Chart

How is COVID-19 changing the use of emergency care by region?

Recent NHS England data show that A&E visits in April 2020 were the lowest since records began. This...

Kjell-bubble-diagramArtboard 101

Work with us

We look for talented and passionate individuals as everyone at the Health Foundation has an important role to play.

View current vacancies
Artboard 101 copy 2

The Q community

Q is an initiative connecting people with improvement expertise across the UK.

Find out more