Improvement Analytics: a new resource in the improvement toolbox

The NHS has seen a succession of change initiatives. Most significantly in recent years, The Five Year Forward View outlined several new care delivery options, including Multispecialty Community Providers and Primary and Acute Care Systems.

But many people will remember programmes such as Evercare, the Whole Systems Demonstrator, Integrated Care Pilots, and Integrated Care Pioneers. With such shifting programmes, it can sometimes feel like groundhog day, and it is commonplace to wonder what has been learnt along the way. But such a view overlooks much of the important detail. Progress has been made in both the care delivery, and the underlying approaches to improvement.  

This week we announce the creation of the Improvement Analytics Unit, a joint initiative between the Health Foundation and NHS England, which will provide rapid feedback on the progress being made by local health care projects in England to improve care and efficiency. The unit will take advantage of the steady advances that have been made in analytical methods, learning from previous national programmes.

These analytical methods are critical to show the impact of complex changes to health care on patient outcomes and cost. They capitalise on an important asset of our health care system - its data.

The figure below examines a telephone health coaching service that began in Birmingham in 2006 and is an example of the kinds of methods that will be used. In this study, existing NHS data sets were used to track the hospital admissions of people receiving health coaching. This data linkage was done in a way that maintained the confidentiality of patient records.

The blue line shows how many emergency hospital admissions were experienced by people receiving telephone health coaching. The control group (red line) shows what might have happened in the absence of health coaching. In this particular instance, the new service did not reduce hospital admissions, and indeed might have stimulated demand.

The controls were selected by an algorithm that ensured they were similar to the intervention patients in terms of their age, health conditions, prior service use, predictive model score, and other characteristics.

Matched controls have been used in at least 30 studies in England, and its methodological underpinnings date back to the 1970s. So although this method has limitations, these are well understood. For example, there may be unobserved differences in the characteristics of the intervention and matched control patients, but the potential impact of these can be assessed.

The question is, now that these analytical techniques have been developed, how can they be spread throughout the NHS, in a way that helps to accelerate change and improvement?

Up until now, these methods have been used predominantly within research studies. These studies have been rigorous, but we have been told by providers and commissioners that the results often come too late. The analysis has also been done as a ‘one-off’ exercise, when improvement needs a drip feed approach, so that changes can be made along the way as appropriate.

The NHS is home to many innovative analysts, but often they do not have the time or resources to invest in new methods, and matched controls are challenging to implement.

That is why we are partnering with NHS England to establish the Improvement Analytics Unit, which will use advanced analytical methods such as matched controls to feedback to a small number of local teams.

For the next three years, the unit will be run jointly by a team of analysts from the Health Foundation and NHS England. In 2016, we will do some initial feasibility testing with a small number of pilots. Then, from 2017, we will begin to feed back information quarterly to a larger number of local teams. From 2019, NHS England will take lead responsibility for the new unit.

The partnership approach recognises that the diffusion of data analytics, like any technology, rests on relationships. Methods like matched controls have been developed largely within academia, and in this project, the Health Foundation is acting as a bridge between those developments and the NHS. Evaluation aficionados will know that this is a fast developing area, and matched controls are just one example of the statistical methods that could be applied.

While the Improvement Analytics Unit is seeking to spread these analytical advances, they need to be seen in the wider context.

While the NHS data sets are valuable, information on patient experience, patient-reported outcomes, and process metrics are generally not collected and linked at patient level. Progress still needs to be made on measurement, in parallel with these kinds of initiatives.

More fundamentally, we know from our history of supporting improvement in the NHS that successful change requires many ingredients.Yes, robust and meaningful data are essential. But so are relationships, a learning environment, and insight from qualitative studies. Attention needs to be placed on the environment in which change happens, as well as the specific interventions.

The Health Foundation is working on several of these fronts, which you can read about in this article from our June 2016 newsletter.

Like the new models of care themselves, improvement analytics is a long-term project. But the prize is a new unit that will help the NHS in England and its partners to gain a better understanding of whether new, local initiatives are improving patient care and supporting the efficient use of resources.

Adam Steventon is Director of Data Analytics at the Health Foundation.