Unwarranted variation: what can we learn from grommets?

With tighter funding and rising demand for health care, clinical commissioning groups are under pressure to get more benefit out of each pound spent on health care. But where to start? Tackling the widespread geographic variations in rates of treatment looks like a good place to me.

Evidence of variations is often interpreted as a sign of ‘overuse’ – that is, where reductions in rates of treatment could release resources alongside gains in health. But there is a big puzzle: how do we know which variations are ‘unwarranted’?

John Wennberg – whose pioneering work in the United States 40 years ago started research into small area variations – argues that unwarranted variation is variation that cannot be explained on the basis of illness, patient preference or medical evidence about effective care.

The usual way to identify unwarranted variation is to do a clinical audit of care delivered and assess what proportion of care was inappropriate (based on well-defined criteria). Clinical audits are an essential tool of evidence-based medicine, but by focusing only on patients who are treated, they will tend to find ‘overuse’. By their nature, they cannot identify the scale of ‘underuse’: that is, patients who would benefit but are not treated. Our research at LSE and UCL, part-funded by the Health Foundation, has developed a way of doing this...

For any given procedure, ask yourself how many patients in a population would be expected to benefit from that procedure. Compare this with the number of procedures that were actually provided. That gives you an indication of possible ‘underuse’ (when the expected exceeds the observed) or possible ‘overuse’ (when the observed exceeds the expected).

We did this for the provision of ventilation tubes (grommets) for otitis media with effusion (OME). The NHS Atlas of Variation showed an eight-fold variation in rates of treatment with grommets across 151 primary care trusts in England. Clinical audits in the UK and the US have found that grommets were inappropriate in two out of every three cases where they were used, suggesting substantial ‘overuse’.

Our team of researchers from LSE and UCL brought together experts in general practice, audiology, paediatrics, ENT, epidemiology and health economics. We developed a model to determine how many children across the population were likely to have OME and would be expected to benefit from grommets (based on NICE guidance).

We found that, even after allowing for uncertainty, our estimates of the number of children in England who would be expected to benefit from grommets far exceeded the number of operations done – indicating substantial underuse. This is an important clinical finding for OME. A report published last month by the Royal College of Surgeons found that 23% of clinical commissioning groups did not follow NICE guidance or had no policy at all on surgical treatment for OME.

Our approach could enable commissioners to work with clinicians and families to remedy the gap between expected and actual procedures and, more generally, to explore together the findings from the NHS Atlas of Variation to consider answers to two key questions:

• How can we increase the appropriateness of patients being referred and treated and reduce both ‘overuse’ and ‘underuse’?
• How can we improve the information available to patients about their treatment options and engage them in shared decision making?

Our research showed that we need to go beyond clinical audit to better understand, at a population level, how many people will benefit from treatment. If the NHS is serious about tackling unwarranted variation, this will be worth the effort.

Laura is a PhD student at the London School of Economics and Political Science