How statistical “magic tricks” promote over-screening

October 24th, 2018

On this blog, we frequently highlight the potential harms of overuse in cancer screening. Earlier this year, participants at the Right Care Alliance Teach-in learned more about cancer screening overuse from Dr. Jill Wruble, DO, radiologist and Assistant Professor at Yale University. Wruble has been speaking to audiences of clinicians, patients, and students (including a TedX video, below) to debunk common misconceptions about cancer screening statistics, what she calls “statistical magic tricks.” 

Here are a few facts about these magic tricks from the video:

The growing cancer epidemic. 

We often hear about how the incidence of cancer has grown over the past decades, and indeed the incidence has grown. But that doesn’t necessarily mean that more people now are being diagnosed with harmful cancers. Wruble points out that although incidence of many cancers has grown, cancer mortality actually dropped 20% over the last two decades. How can this be?

Wruble says it is because not all cancers are harmful — some types of cancers never grow or cause symptoms. You might say that we’re facing an “epidemic” of harmless cancers. With a few exceptions (colon, cervical, and possibly lung cancer), early detection has not reduced cancer mortality in asymptomatic low-risk patients, says Wruble. 

False positives are more common than you think.

If a 40-year-old woman has a positive result on a mammogram, what is the likelihood that she actually has cancer? You would think the number would be high, around 80 or 90 percent. But it’s actually only 8.3%. The reason is because the incidence of cancer is very low, so even a small likelihood of false positives means that most positives will be false. 

Out of 10,000 40-year-old women, about 100 have cancer and 9900 do not. With a false positive rate of 10%, this means that 90 women have a true positive but 990 have a false positive. So out of 1080 positive tests, 990 (91.7%) will be false positives. If a disease is rare, most positive tests will be false positives. “It’s counter-intuitive, but it’s true,” says Wruble.

Relative risk is not the same as absolute risk.

It makes sense to support cancer screening when we hear they reduce deaths from lung cancer by 20% or reduces breast cancer mortality by 33%. However, these statistics are less impressive when you look at the absolute risk, not the relative risk. For example, if an intervention lowers mortality from a disease from 60% to 36%, that’s a 40% relative reduction and a large impact. But if an intervention decreases mortality rate from 5% to 3%, it’s still a 40% relative reduction, but a much smaller impact. Without knowing the absolute numbers, it’s difficult to understand the benefit of the intervention. 

Wruble recommends that patients review decision aids that clearly outline the harms and benefits of cancer screening in absolute terms, so that they and their loved ones can make decisions based on their own values.

For more on seeing past “statistical magic tricks,” watch the full video of Wruble’s talk!