Why Two Oncologists May Recommend Different Treatments

The most direct answer comes from NCI's own summary on levels of evidence. It describes a formal ranking system. Then it states something plainly. Depending on perspective, different expert panels, professional bodies, or individual physicians may use different cut points of overall strength of evidence. That applies both to writing guidelines and to taking action.

That is the mechanism in one sentence. Two oncologists can read the same trial. They can agree on what it found. And they can still disagree on whether it is strong enough to change practice. Neither is ignoring the evidence. They are applying different thresholds to it.

NCI adds why the formal ranking still matters. A written description of the level of evidence gives everyone a uniform framework for the data. That framework is what makes specific recommendations possible at all.

PDQ ranks each result on two separate scales, then combines them. The first is the strength of the study design. The second is the strength of the study end point. That means what was actually measured.

Study designs, strongest first:

• Randomized controlled trials. The double-blinded randomized trial is the gold standard. NCI notes a practical limit. Most oncology trials cannot be double-blinded after treatment is assigned. Procedures and side effects differ so plainly that both patient and clinician can tell.
• Meta-analyses of randomized trials, which pool earlier studies.
• Nonrandomized controlled trials. Subset analyses inside randomized trials often land here.
• Case series and other observational designs. NCI calls these the weakest form.

End points, strongest first:

• Overall survival from a defined time. NCI calls this arguably the most important outcome to patients. It is also the most easily defined, and the least open to investigator bias.
• Cause-specific mortality. Possibly the most biologically important. But it is more subjective, and more open to investigator bias. It can also miss effects of therapy that shorten overall survival.
• Carefully assessed quality of life. This matters greatly to patients. NCI says careful documentation of it, within a strong design, is enough for most physicians to adopt a treatment.
• Indirect surrogates. These are event-free survival, relapse-free survival, disease-free survival, progression-free survival, and tumor response rate.

Combining the two scales produces short labels. A1 is a randomized trial with an overall survival end point. B1 is a randomized trial with a disease-free or progression-free survival end point. C2 is a case series with a progression-free survival end point.

Here is where two doctors part company. NCI writes that surrogate end points may translate into direct patient benefit. That means survival or quality of life. But they do not automatically do so. NCI then says something important. It is rational in many circumstances to use a treatment that improves a surrogate end point, while waiting for a more definitive one.

"Rational in many circumstances" is not the same as "required in all." One oncologist may act on a progression-free survival gain. Another may wait for overall survival data. Both positions hold up under the same framework.

Ovarian cancer supplies a documented case. A Gynecologic Cancer InterGroup meta-analysis pulled individual patient data from 17 randomized trials. Each had at least 60 newly diagnosed patients. They were published from 2001 through 2016. Its conclusion was blunt. Progression-free survival is not an adequate stand-in for overall survival in that disease.

Cervical cancer supplies another. The KEYNOTE-A18 trial tested chemoradiation with or without pembrolizumab. Progression-free survival improved significantly, at a hazard ratio of 0.70, interval 0.55 to 0.89. Overall survival at 24 months was 87% against 81%, at a hazard ratio of 0.73. But that interval ran 0.49 to 1.07, which crosses 1.0. Two clinicians can weigh that pair of results differently, and both be reading it correctly.

Large real-world databases feel authoritative. NCI's summary describes a study that tested how well they actually predict trial results.

Investigators searched for observational studies published from 2000 to 2016. Those studies used SEER, SEER-Medicare, or the National Cancer Database to compare cancer treatment regimens. The team matched 350 treatment comparisons to 121 randomized trials making the same comparison.

The results were sobering. There was no significant correlation between the two sets of hazard ratios. The concordance correlation coefficient was 0.08, with a 95% confidence interval of minus 0.07 to 0.23. Only 40% of matched studies agreed on treatment effect, a kappa statistic of 0.037. Only 62% of the observational hazard ratios fell inside the matched trial's 95% confidence interval. None of the correlations beat what chance would produce. And they did not improve in studies using the most advanced statistical methods. Those methods included propensity score weighting, instrumental variable adjustment, and sensitivity analysis.

One figure stands out. Take the 70 observational studies ranked as rigorous that reported overall survival. Of those, 35 reported a positive result where the randomized trial found no difference, or an effect in the opposite direction. That is half of them.

Meta-analyses are not immune either. NCI cites a comparison of single large randomized trials against earlier meta-analyses of smaller trials on the same topics. Agreement was only fair, at a kappa statistic of 0.35. The large trial's outcome was not predicted accurately 35% of the time. Meta-analyses by different investigators on the same question can reach opposite conclusions. That is why PDQ places meta-analyses of randomized trials in the same or a lower category than randomized trials. Never higher.

The PDQ summaries do not paper over open questions. Reading them shows how much real uncertainty sits inside standard practice.

• On preventive lymph node dissection in penile cancer, NCI states the effect on survival is not known. It says opinions vary on its use.
• On DPYD genetic testing before fluoropyrimidine chemotherapy, NCI calls the issue controversial. It says the question needs further study. Testing costs under $200, but may delay therapy by about 2 weeks.
• On intermediate Oncotype DX recurrence scores in breast cancer, above 11 and up to 25, NCI calls the chemotherapy decision complex and personalized. It takes in age, clinicopathological features, and patient preference.
• On follow-up after colorectal cancer surgery, NCI says no high-level evidence exists to guide surveillance. It adds that no large randomized trial has shown a survival benefit for standard monitoring programs.

None of those are gaps in one doctor's knowledge. They are gaps in the evidence, and different clinicians fill them differently.

The useful move is not to pick the more confident doctor. It is to make the reasoning visible. These questions do that.

• What level of evidence supports each recommendation? Is it a randomized trial with an overall survival end point, or something less?
• Which end point improved in the trial being cited: overall survival, or a surrogate such as progression-free survival?
• Why do the two plans differ? Is it different evidence, a different threshold for acting on the same evidence, or a different read on my situation?
• Would a tumor board review resolve this, and can it be requested?
• Does either plan foreclose the other? Which decision is reversible?
• Is a clinical trial available at one center and not the other, and is that driving the difference?

Also see how to get a second opinion for cancer for the mechanics, and What Happens After a Cancer Diagnosis for the earlier conversation.

• NCI PDQ — Levels of Evidence for Adult and Pediatric Cancer Treatment Studies
• NCI PDQ — Ovarian Epithelial, Fallopian Tube, and Primary Peritoneal Cancer Treatment, health professional version
• NCI PDQ — Cervical Cancer Treatment, health professional version
• NCI PDQ — Penile Cancer Treatment, health professional version
• NCI PDQ — Breast Cancer Treatment, health professional version
• NCI PDQ — Colon Cancer Treatment, health professional version