AI and health care: risk and our imagination shouldn’t hold us back

In his recent REAL Challenge lecture for the Health Foundation, Hetan Shah boiled down the debate on artificial intelligence into a single sentence: ‘There are possible benefits, great benefits, but there are risks…’ We hear much debate about how to mitigate those risks, and rightly so. But I believe there are two key factors limiting our ability to leverage AI that we’re not talking about enough: our imagination and our collective appetite for risk. To realise AI's true potential in health care, we must cultivate both. 

The NHS faces unprecedented challenges, from patient dissatisfaction and workforce shortages to outdated infrastructure. Change is no longer optional – it is a necessity. AI, particularly generative AI (or even artificial general intelligence (AGI) in the future), presents a powerful opportunity to transform the system – if we can harness its potential wisely. So, what do we need?

A test of appetite for risk

One of the clearest current demonstrations of our risk appetite dilemma is whether to introduce autonomous AI for chest X-ray reporting. AI could remove ‘normal’ cases with high confidence and fast track abnormal or uncertain cases for urgent specialist review. While such models show promise in the literature, concerns around liability, legal considerations, regulations and public trust have stalled real-world implementation. Hetan Shah talked about the need to ‘test, learn, evaluate and drop what doesn’t work’, but our current appetite for risk means we are often too worried to learn.

A test of imagination

While Hetan also highlighted that ‘agentic AI’ – which can solve problems with minimal human supervision – may be the next big thing in 5 years’ time, the NHS is still in the early stages of thinking about very narrow applications of existing AI tools. We need to think about moving our use of AI in the future beyond siloed applications towards adaptive systems that evolve with real-time feedback. While current regulations cater to narrow AI models (those programmed to perform a singular or limited specific task), the greatest challenges will require more imaginative, complex systemic and multimodal solutions. Until we experience our ‘AI tipping point' (ie an innovation that fundamentally reshapes how we deliver care, like how digital streaming revolutionised the entertainment industry), AI adoption will remain fragmented and suboptimal.

Tech can’t solve what society won’t fix

Hetan also raised the issue of bias, but instead of highlighting this as an indictment of the technology, let’s view it as a call to action. AI serves as a mirror, exposing inequities embedded in our society and our health system. Fixing our data doesn’t necessarily fix our problems. We must be clear to distinguish between ‘AI/tech’ problems and ‘people/society’ problems, using these insights not only to improve AI but to drive meaningful societal change. In paediatrics, my area of interest, the dearth of child-specific health tech sadly shows us how we still neglect creating digital innovations for many of our most vulnerable patients. In Hetan’s words, ‘You can’t layer AI on a failing service.’ 

Balancing infrastructure and innovation

Discussions about upgrading digital infrastructure are no doubt essential, but should not come with innovation paralysis. Hetan’s lecture highlighted the power of the NHS to shape markets. Leveraging collective NHS buying power and fostering national collaboration could help bridge the gap between cutting-edge AI and the essential foundations that enable it in a cost-effective manner. But we do need a solutions-first approach, defining what good looks like (for both infrastructure and AI) and how human-AI collaboration fits into this framework.

To move forward, we must ensure regulations support, rather than stifle, innovation. Encouragingly, we are already seeing initiatives such as CERSI-AI (Centres of Excellence for Regulatory Science and Innovation) and the introduction of the MHRA AI Airlock (ie regulatory sandbox) systems allowing for controlled testing of novel AI solutions before widespread adoption. However, we need to further promote such mechanisms to accelerate responsible AI integration.

A parallel (more accessible) health care system emerges

While health care AI remains locked in regulatory debates, consumer AI is evolving rapidly, creating a parallel ecosystem outside traditional health care, with patients and clinicians alike turning to unregulated tools. The other day a friend told me they’d uploaded their own MRI scan into a widely known generative AI model to get an expedited (wrong) diagnosis before seeing their doctor. Banning these tools may be tempting, but ultimately futile. If AI is perceived as offering value (ie faster access over medical accuracy for now), some people will use it. We must also prioritise digital literacy and risk awareness to prevent a fragmented, unsafe AI health care landscape for everyone.

The question now is not if, but how we will govern and deploy AI to harness its full capabilities. Regulators are in a golden and coveted position to engage collaboratively to shape the AI we see and implement it for societal benefit, not to simply accept or reject solutions. 

The NHS is a system in crisis – remaining locked in a holding pattern as the system deteriorates is a risk we cannot afford. AI is not the silver bullet, but it could be the lever we need to shift a broken system. By cultivating our imagination and tolerance for risk, and with robust governance, we can harness AI's potential to remake health care from within.