What Patients Should Ask After an AI Cancer Headline

The gap between a headline and a hospital

An AI cancer headline usually reports an accuracy number. A model matched or beat clinicians at spotting something on a scan or in a record. That is a real result. On its own, it is still a long way from a tool anyone can use in a clinic.

This page sets out the questions that close that gap.

Where AI already sits in cancer care

FDA keeps a public AI-Enabled Medical Device List. It names devices that use artificial intelligence and are cleared for marketing in the United States. Each entry links to FDA's own database record, with summaries of safety and effectiveness.

FDA says the devices on it have met applicable premarket requirements. That includes a focused review of overall safety and effectiveness. It also includes a check that the studies suited the device's intended use.

Two caveats come from FDA directly. The list is not a complete record of AI-enabled devices. Entries were found largely by searching for AI-related terms in authorization summaries. And the published summaries are not all-inclusive.

Scan the recent entries and a pattern jumps out. Most are radiology devices. In cancer, AI is furthest along in reading images, not in choosing treatments.

Validation: the question that matters most

A model trained at one hospital learns that hospital. It learns its scanners, its patient mix and its labeling habits. Testing it on data held back from that same source is called internal validation. It is the easiest test to pass.

External validation means testing on data from a different hospital, with different equipment and a different population. Performance usually drops, sometimes sharply. If a headline does not say which kind of validation was done, ask.

The strongest evidence is prospective. The tool runs in a real clinic, in real time, and outcomes are tracked going forward. Most published AI results are retrospective. The model was run over records that already existed.

Accuracy is not benefit

A tool can be more accurate and still leave patients no better off.

Think about what an imaging model's output sets off. More findings flagged means more follow-up scans, more biopsies and more worry. If the extra findings are cancers that would never have caused harm, the tool has produced overdiagnosis, not benefit.

One question separates the two. Did the tool change anything that matters? Fewer missed cancers, an earlier stage at diagnosis, fewer needless procedures, better survival. Accuracy on a benchmark answers none of those.

Our page on what clinical trials are explains why measuring outcomes needs a different study design from measuring performance.

Errors are not distributed evenly

An overall accuracy figure is an average, and averages hide the cases that matter.

Ask how the model did in specific groups. By age. By sex. By race and ethnicity. By skin tone, for anything in dermatology. By breast density, for mammography. By body size, and by scanner brand. A model trained mostly on one population can be clearly worse on another while its headline number holds up.

Ask about the shape of the errors too. A missed cancer and a false alarm are both errors, and they cost very different things. A tool tuned to miss almost nothing will flag a great deal.

Who is accountable, and does the tool change?

Two structural questions are worth asking about any AI tool in a clinic.

First, is it helping a clinician or replacing a step? A tool that flags images for a radiologist to review is in a different safety position from one that clears images with no human looking.

Second, does the model change over time? Machine learning systems can be updated as they meet new data. FDA has published guiding principles on change control plans for such devices. A device that changes after authorization raises questions a fixed device does not.

FDA, Health Canada and the UK's Medicines and Healthcare products Regulatory Agency published ten guiding principles for Good Machine Learning Practice in October 2021. The International Medical Device Regulators Forum built on those in a final document in January 2025.

Questions to bring to an appointment

• Is this tool used at my center, or is it still research?
• Was it tested outside the dataset it was built on?
• Did it improve patient outcomes, or only accuracy scores?
• What kind of errors does it make most often?
• Was performance checked in people like me?
• Does a person review its output before it affects my care?
• What happens to my scan or record if the tool disagrees with the radiologist?

Our pages on getting a second opinion and questions about your cancer diagnosis cover how to raise these without friction.

What this story cannot tell

• It cannot tell whether an AI headline applies to any particular person's cancer or care.
• It cannot replace a treatment plan from a care team.
• It cannot show that an AI result is better than a clinician's judgment in a specific case.
• It cannot support starting, stopping or changing treatment.
• Authorization means a device met a premarket standard for its stated use. It is not a promise of benefit in every clinic or for every patient.

Sources

• https://www.fda.gov/medical-devices/software-medical-device-samd/artificial-intelligence-enabled-medical-devices
• https://www.fda.gov/medical-devices/software-medical-device-samd/good-machine-learning-practice-medical-device-development-guiding-principles
• https://www.fda.gov/medical-devices/software-medical-device-samd/artificial-intelligence-and-machine-learning-software-medical-device
• https://www.cancer.gov/research/participate/clinical-trials

An AI-assisted editorial system helped prepare this page. No named medical reviewer has reviewed it unless one is listed.

See an error, old source, or unclear wording? Tell us.