A global analysis found that an artificial intelligence (AI) system was able to predict breast cancer more accurately than humans.
“Screening mammography aims to identify breast cancer at earlier stages of the disease, when treatment can be more successful,” the study authors wrote. “Despite the existence of screening [programs] worldwide, the interpretation of mammograms is affected by high rates of false positives and false negatives.”
The study team included researchers from Google’s AI team as well as researchers from DeepMind, Cancer Research UK Imperial Centre, Northwestern University, and Royal Surrey County Hospital.
The evaluation, published in the journal Nature, included one U.S. dataset with more than 15,000 women and one UK dataset with more than 76,000 women. The AI system was then further tested for its ability to screen a de-identified dataset, with more than 25,000 UK women and more than 3,000 U.S. women. The AI system was associated with an absolute reduction in false positives of 5.7% in the U.S. database analysis and 1.2% in the UK database analysis; and an absolute reduction of 9.4% and 2.7%, respectively, of false negatives.
In order to determine whether the AI system could generalize different healthcare systems, the team trained the model using only the UK-based data and tested the model on the U.S. dataset. The AI system yielded a 3.5% reduction in false positives and 8.1% reduction in false negatives.
AI Trumps Human Experts
The authors independently compared the AI system against six radiologists, and the AI system was more accurate in each instance; the area under the receiver operating characteristic curve (AUC-ROC) for the AI system, compared to the AUC-ROC of the average radiologist, was greater by an absolute margin of 11.5%.
A press release from Google noted that the AI system had less information than the experts in the human versus AI comparison: “The human experts (in line with routine practice) had access to patient histories and prior mammograms, while the model only processed the most recent anonymized mammogram with no extra information. Despite working from these X-ray images alone, the model surpassed individual experts in accurately identifying breast cancer.”
According to the American Cancer Society, breast cancer is the most common cancer among women, excluding skin cancers. About one in eight women in the U.S. will develop breast cancer in her lifetime. Breast cancer is the second leading cause of cancer-related death among women, with lung cancer being the first.
