Putting people at the heart of data-driven health systems to address inequalities Reflections on our partnership with the Ada Lovelace Institute

Barely a day goes by without data-driven technologies such as AI and data platforms being in the news. Just in the past month we’ve seen the announcement of a new £100m mission to accelerate the use of AI in life sciences, and the UK government hosted the first global AI safety summit. 

While these technologies have tremendous potential to improve the health of the population, making sure the benefits are realised by everyone – and are designed to reduce health inequalities – is crucial. This has been our focus over the past three years in a research and public engagement partnership between the Health Foundation and the Ada Lovelace Institute. 

Engaging diverse perspectives

In September the Ada Lovelace Institute hosted an event – Tackling inequalities in data-driven systems – to tie together the insights coming out of this work. A key focus of the event was how to better align the priorities of people that design digital health care services with the main needs and concerns of the patients and carers that use them. An accompanying report, Access denied? describes the need to better understand and engage with often-marginalised perspectives when designing, commissioning or developing policy for digital health services. Other initiatives such as the AI Fringe, which complemented the global AI Safety Summit, also aim to bring together the views of civil society, industry and others. 

Incorporating diverse perspectives like this takes time and resources and may be particularly challenging for small organisations. This raises questions about what central support could be made available for this kind of engagement activity. What impact could NHS England’s planned £2m programme of public engagement on the use of health data have here? 

Co-production is feasible

As well as engaging and involving patients and carers, we should be focusing on giving them the skills, knowledge and confidence needed to both better understand how a data-driven health care system works, and to comment, challenge and advocate for themselves. Even on the topic of equitable AI, which is often discussed in highly technical terms, co-production with public groups is feasible. 

Panellists at the event shared examples of how they’d created ‘deliberative knowledge spaces’ to do just this. In this instance, a research team presented key concepts, and a public involvement group asked questions and co-produced thinking about how fairness in algorithms should be defined and which fairness domains should be assessed.

Understanding what's causing data inequalities

The event was also an opportunity to hear about the independent review on equity in medical devices from its chair, Professor Dame Margaret Whitehead. A summary of the evidence, including evidence on equity in AI technologies, and recommendations from the expert panel is due to be published shortly.

It was interesting to hear of the blindness about data input on AI technologies that are approved for use in the NHS and to contrast this with the pharmaceutical approvals process where inclusion criteria and data are independently scrutinised.  

Discussions also covered some of the incentives and barriers to probing inequalities in data-driven technologies. There are potential risks to reputation and to timelines if developers opt to assess the equity of their algorithm or app under the magnifying glass. There is a lack of data on protected and socioeconomic characteristics that would allow differences across population segments to be assessed and continually monitored. There is not currently a universal definition of algorithmic fairness or an agreed set of metrics to report on. These are just some of the challenges. 

What role could regulation play in levelling the playing field and clarifying the requirements for equitable data-driven technologies? These, and other challenges, along with recommendations have been summarised by the Ada Lovelace Institute in a policy briefing. 

Showing it can be done

Happily, the event was also an opportunity to hear from several teams pushing forwards in this space, including two projects we have been delighted to support at the Health Foundation. 

Public Health Scotland have assessed their SPARRA risk prediction algorithm to understand how well it performed across different demographic groups. They have now made the code and package openly available so that others can make use of this tool for assessing fairness.

Another project, STANDING Together (supported by the Health Foundation in partnership with the NHS AI Lab), has developed standards for diversity and inclusivity in health care data through an international consensus process.

The work we have been supporting, with partners like the Ada Lovelace Institute and the NHS AI Lab, has only scratched the surface on some of these issues. There is much more to do. 

Alongside continued focus on scaling AI and data-driven technologies across health and social care, we need to maintain an equal focus on ensuring that people are at the heart of these efforts – something which we will be looking to contribute to further through our research, analysis and funding programmes.

Josh Keith (@joshkeith) is Assistant Director of Data Analytics at the Health Foundation.

Mai Stafford (@stafford_xm) is Senior Analytical Manager at the Health Foundation.

This content originally featured in our email newsletter, which explores perspectives and expert opinion on a different health or health care topic each month.