The era of big data?

Returning from the holidays, my inbox is crammed full of emails announcing the ‘era of big data’. Much like celebrating the new year, these proclamations can seem arbitrary – applied health services research and epidemiology have been using large data sets for decades, and the work of tomorrow will be a continuation of this past, rich tradition. But nevertheless, at least 25 data science masters degree courses have now sprung up in the UK, producing thousands of capable analysts every year, and recently hundreds of £m have been invested in big data centres and infrastructure for data linkage.

Although it is still unclear whether ‘big data' constitutes a fad or an era, there is the opportunity to use the energy that exists at the moment to make meaningful change to how health and social care is provided.

The history of big data

Analysis of large administrative databases can be traced back at least as far as the 1950s, with epidemiological work using a nationwide database of linked records collected by the Veterans Health Administration in the United States.

Inspired by the approach, Acheson and others gathered data on hospital stays, births and deaths for around 18,000 residents of Oxfordshire, storing the linked records, in the first instance, on punched cards. Also, in 1954, Doll and Hill published their study of death records linked to self-reported smoking status for almost 25,000 doctors in England, showing an association between smoking and lung cancer.

Data analytics has been gathering momentum for decades, owing to improvements in computing, electronic medical records, and developments in statistics and econometrics. But the relationship between evidence and action has never been straightforward (just look at smoking rates).

Turning evidence into action

Recent years have seen the routine linkage of GP and hospital data at person level in many areas in the UK. These offer deeper clinical insights than administrative data alone, and there is potential for new approaches to commissioning, population health management, case finding and evaluation.

Particular attention has been paid to the use of predictive risk models to identify cohorts of at-risk patients. Retrospectively matched control groups are being used more often to evaluate the impacts of complex interventions, and these methods also benefit from linked data sets. The Byzantine complexity of information governance rules often seems to be a rate limiting step, but at least in recent years there has been a more active debate about how privacy can be balanced against the public good.

All this represents progress with data linkage and analysis, but how can we build on it and make the most of the surge of interest in data analytics to improve care?

Three changes are needed if data analytics is to meet the need for improved health system performance.

1. Use information to change practice

First, the yawning gap between analysis and decision making must be narrowed. Friedman distinguishes between two parts of a learning health care system, the part that gathers information through analysis, and the part that uses this information to change practice.

These can work better together but there are challenges at all levels of the health care system – quality improvement efforts within clinical microsystems, commissioning, and national policy making. The problems are not only about volume, variety and velocity – the ‘three v’s’ of big data. There is too often a disconnect between the metrics studied and the dimensions of care valued by practitioners and service users, and collaboration between analysts and other people interested in quality improvement can be deepened.         

2. Improve how we capture the perspective of service users

So, the second priority is that we need to get better at capturing the perspective of the service user within routine data sets, including when they are away from the clinic, for example by continuing to explore the potential of modern communications technology.

Routine data are heavy on utilisation and cost metrics, meaning that we might prioritise these disproportionately compared with what matters to patients. Plenty of candidate metrics exist but the problem is how to use them effectively. Efficiencies can be gained from using the same data at each level of the health care system – so that, for example, the metrics analysed to inform commissioning parallel those targeted within the clinical consultation.

3. Create a thriving analytical community focussed on quality improvement

Finally, the analytical workforce needs to be become more integrated, able to engage with quality improvement in the right spirit and with the right backing.

Our best guess is that around 20,000 people use statistics as part of their day-to-day job in the NHS in England, most of whom are focused on financial management rather than improving quality more generally.

What would a thriving analytical community focussed on quality improvement look like? How can the analytical workforce be supported to develop and to share ideas? How can collaboration be encouraged not just between analysts, but also with qualitative researchers, psychologists, and sociologists?

How are we making this happen?

Our new data analytics team is active in all three areas. For example, we’re planning new work to compare the outcomes of patients receiving new forms of care with those of matched control groups. By doing this at scale, we hope to provide more rapid information for improvement.

We have also just linked data on primary and secondary care activity with patient activation scores – a measure of a person’s willingness and ability to manage their own health care. And, in 2016, we are scoping what contribution we can make to support analysts working within the NHS.    

One of the key determinants of progress will be the ability to try new things, to experiment and to innovate. The Health Foundation’s grants to frontline NHS teams across the UK provide opportunities. With further experimentation we will better understand the contribution that data analytics can make across the NHS.

Adam Steventon is Director of Data Analytics at the Health Foundation.