From chatbots to clinics: staying ‘SMART’ about AI for health care

An interview with Joseph Alderman

What are AI chatbots and how are they changing access to health information?

Chatbots are not just apps like ChatGPT. When you Google something now, you get an AI response. That’s a chatbot. You can talk to it and ask further questions. More people are already using chatbots than even realise it. 

And more and more people are now using chatbots to ask questions about health. We will be releasing a new report soon that shows the number of people regularly using AI in this way is growing really fast, reflecting how the tools we use to find information are shifting.

With some obvious caveats, patients having access to AI to ask questions about their health can only be a good thing. For years people have used Wikipedia or Google to research health questions. Some health professionals don’t like that, almost like seeking out information is forbidden. That's dismissive, paternalistic and unhelpful. Of course, the other equally unhelpful extreme would be to strongly recommend people use chatbots without carefully discussing the limitations of such technology.

A more rational approach is to recognise the huge potential here. If people come to an appointment having already done some reading (and frankly, if someone spends half an hour with an AI chatbot, they probably know more about the disease they're talking about than I do) that’s a really good thing. That means my job has stopped being just giving out information and is now helping people to understand and contextualise that information, and also to correct things that are wrong (because the caveat here is that it’s only helpful if the information they’re getting is any good).

Is it safe to ask AI for health information? 

There have been some high-profile examples where chatbots have given people information which isn't true, or in some sad cases, given guidance that has been very harmful. The most tragic examples are when people have been experiencing disordered thinking or suicidal thoughts. That happens because these chatbots have a predilection to sycophancy; they tell us what they think we want to hear. 

I try to conceptualise risk and safety across multiple axes. It’s all about context. If someone's experiencing a medical emergency or a mental health crisis, then far and away the safest thing to do is seek urgent medical advice from a professional, not speak to a chatbot. 

Some groups are also more at risk of not noticing when chatbots are giving them the wrong advice – this could include people who are newly diagnosed with a condition, people with a learning disability, or those whose first language isn’t English. 

But when it comes to risk, we must remember that the current system is also far from perfect. Patients can fall between the cracks of our knowledge and of our services. Even in the UK there are people with very poor access to health care, including those in rural communities or from disadvantaged groups. These people aren’t always well served by the health service as it exists now.

If I have access to ‘super intelligence’ in my pocket, well, maybe that gives me the ability to do or know things that otherwise would have required a visit to the doctor. So sometimes these tools are risky, and other times they are powerful enablers for people who don't currently have good access to health care. 

How are chatbots influencing demand on the NHS? 

GPs I speak to say they're seeing patients come to them with questions that are being raised by chatbots. What we don't know is whether those patients were always going to visit their GP. And for every patient who goes to the GP after speaking to a chatbot, there might be someone else who had their questions answered and didn't need to go. Researchers at King's College London found that 1 in 7 UK adults have used a chatbot rather than seeing their GP or another health professional, so there's at least some evidence of demand switching. 

Tell us about the guide you’ve developed to help people use health chatbots safely.

The Health Chatbot User Guide is a practical guide for anyone using AI to ask health questions. It’s not about encouraging people to use AI for health advice, it just acknowledges that people already do, and that we want people to know how to do it safely. 

It’s written in plain English, and guides people to ask SMART:

S –  slow down and think before asking health questions to an AI chatbot

M –  make your questions clear

A –  ask yourself: ‘What am I sharing?’

R –  remember to check the facts

T –  talk to a real person when you need to.

It goes into detail on all of those areas, giving advice for how to write prompts, how to specify what sources should be used, explaining how information you share with an AI chatbot might not stay private, and ensuring people understand that although chatbots might sound like a human, they work like a machine. 

We built the guide predominantly to be useful for patients, but we hope that health professionals and parts of the health system can also adapt some of the wording and start to use it in their communication with patients. 

You’re also involved in work on AI readiness in the NHS, what are the factors that make an organisation ready to introduce AI into their health care pathway?

Our team at the University of Birmingham and University Hospitals Birmingham have been building an AI readiness checklist and an assurance framework to help NHS organisations understand their readiness to deploy AI into health services. This is work funded by NHS England and the Health Foundation.

Rather than interpreting readiness as a binary state, our framework sees it as a continuum, moving from ‘definitely not ready’ to ‘100% ready’. 

Depending on what the technology is, the clinical problem that we're dropping it into, and the organisation itself, there'll be a different level of readiness needed. Something administrative and quite straightforward might require a lower state of readiness than something semi or fully autonomous that replaces clinical decisionmakers and therefore has a higher level of risk. 

The checklist is currently published as a draft for consultation and we’re now having conversations about how it will get implemented across the ecosystem.

When does using AI in health care work best? 

All technology works best in health care where it solves real problems. Too often we’ve deployed digital technologies seemingly for the sake of it, rather than because it's clearly going to help. 

I'm a real believer that for us to see the benefit from AI in health care, we need three things to overlap. First, we need to be working on real problems that real people are experiencing – be that patients or health professionals. Second, it needs to overlap with the things that AI is actually good at, because we know AI is not good at everything right now. And third, we need to choose things that the system is truly ready for.

That overlap is often quite small. My worry is that if we're short on any one of those domains, we'll end up with tools that either don't work or don't help.

But, if we can find those sweet spots and target our development there, we're likely to produce and implement tools that really help people. 

This content originally featured in our deep dive newsletter.