Doctors, as a whole, are a pretty clever bunch, but they can be resistant to change. The most famous example is probably the 19th-century surgeons who refused to wash their hands when moving from mortuary to labour ward, spreading as-yet-undiscovered microbes and leading to infant deaths. Hungarian physician Ignaz Semmelweis, who collected statistics to make the case that soap and water could save lives, was ridiculed and ostracised.

Today, we live in more enlightened times, and medical practice is generally backed by evidence – but are we always getting the right evidence to bring about change? For example, there are signs that bringing artificial intelligence into clinical use could also save lives. As we report in “AI helps radiologists spot breast cancer in real-world tests”, radiologists who chose to use an image-classifying AI to help spot breast cancer picked up an extra case per 1000 people screened. Across healthcare systems, the effect could be big.

Does that mean we should encourage doctors to hang up their scrubs and let the machines take over? Far from it. While large language model AI systems like ChatGPT can ace multiple-choice medical tests, they do less well on conversational diagnoses (see “AI chatbots fail to diagnose patients by talking with them”). A medic with a good bedside manner and listening ear is still vital, for now.

We should be bolder in testing medical AI systems in real-world settings

Instead, there are two conclusions we can draw from these studies. The first is that we should be careful about using the generic term “artificial intelligence”. Although the two systems we report on share an underlying neural network technology, image classification is a very different task to text generation, and the latter has a much higher risk of the AI spitting out plausible but incorrect results. In other words, not all AIs are made equal.

The second conclusion is that we should be bolder in testing medical AI systems in real-world settings, rather than just in the lab or simulations. The breast cancer study, by giving radiologists control over when to use AI, shows it can be a useful tool. With a push to get more evidence like this, lives could be saved, just as after Semmelweis, who is now considered a medical hero.

Topics: