Rob Dillard

DocwireNews

A combined approach of crowd innovation and artificial intelligence (AI) algorithms can rapidly produce tumor targeting results for radiation oncologists, according to a <a href="https://jamanetwork.com/journals/jamaoncology/fullarticle/2730638">study</a> published in JAMA Oncology.To complete this study, researchers conducted a 10-week, prized-based, online, three-phased challenge. A total of 564 contestants registered for this challenge, 34 of whom submitted 45 algorithms. These were scored on an independent dataset, which the research team withheld from contestants. They rated the performance of each algorithm using quantitative metrics that assessed any overlap in algorithmic automated segmentations with expert&rsquo;s segmentations. A clinical care expert generated a carefully-selected dataset for the contest, including computed tomography (CT) and lung tumor segmentation scans from 461 patients (median images per scan, 157 images per scan; 77,942 images in total; 8,144 images with tumor present).In order to develop the algorithms, each contestant was supplied a training set of 229 CT scans with accompanying expert outlines while being provided with feedback on their performance through the duration of the contest from the study&rsquo;s expert clinician.The contest took place on Topcoder.com, a commercial platform that hosts thousands of online algorithm challenges for programmers to decode while competing for prizes. The participants learned about the underlying medical problem via online video and written materials showing the clinician demonstrating the manual lung tumor segmentation.During each of the three study phases, the contestants used their algorithms to produce segmentations on the validation set and underwent real-time evaluation of their algorithm&rsquo;s performance based on the segmentation scores (S scores), which were made visible on a public leaderboard. At the conclusion of each phase, prizes were issued to the contestants whose algorithms generated the highest S score.<h2>Algorithms Prove Accurate </h2>According to the study results, the automated segmentations produced by the top five AI algorithms, when combined using an ensemble model, showed an accuracy (Dice coefficient = 0.79) that was within the benchmark of mean interobserver variation assessed between six human experts. In the first phase, the top seven algorithms showed an average custom S scores on the holdout data set ranging from 0.15 to 0.38, and suboptimal performance using relative measures of error.Moreover, the mean S scores for phase two increased to 0.53 to 0.57, displaying a similar improvement in other performance metrics. Finally, for the third phase, performance of the top algorithm increased by an additional 9%. Overall, using an ensemble model and combining the top five algorithms from phases two and three yielded an additional 9% to 12% improvement in performance with a final S score reaching 0.68.<blockquote class="twitter-tweet" data-width="500" data-dnt="true">
New paper on <a href="https://twitter.com/JAMAOnc?ref_src=twsrc%5Etfw">@JAMAOnc</a> from <a href="https://twitter.com/DanaFarber?ref_src=twsrc%5Etfw">@DanaFarber</a> <a href="https://twitter.com/BrighamWomens?ref_src=twsrc%5Etfw">@BrighamWomens</a> <a href="https://twitter.com/HarvardHBS?ref_src=twsrc%5Etfw">@HarvardHBS</a> &amp; <a href="https://twitter.com/LISHarvard?ref_src=twsrc%5Etfw">@LISHarvard</a> explores how <a href="https://twitter.com/hashtag/crowd?src=hash&amp;ref_src=twsrc%5Etfw">#crowd</a> <a href="https://twitter.com/hashtag/innovation?src=hash&amp;ref_src=twsrc%5Etfw">#innovation</a> can be used to develop <a href="https://twitter.com/hashtag/AI?src=hash&amp;ref_src=twsrc%5Etfw">#AI</a>-based solutions to potentially improve <a href="https://twitter.com/hashtag/cancer?src=hash&amp;ref_src=twsrc%5Etfw">#cancer</a> <a href="https://twitter.com/hashtag/care?src=hash&amp;ref_src=twsrc%5Etfw">#care</a> cc: <a href="https://twitter.com/Dr_RayMak?ref_src=twsrc%5Etfw">@Dr_RayMak</a> <a href="https://twitter.com/klakhani?ref_src=twsrc%5Etfw">@klakhani</a> <a href="https://t.co/Dh49fjfwbo">https://t.co/Dh49fjfwbo</a>
&mdash; Laboratory for Innovation Science at Harvard (@LISHarvard) <a href="https://twitter.com/LISHarvard/status/1118899068565168130?ref_src=twsrc%5Etfw">April 18, 2019</a>
</blockquote><script async src="https://platform.twitter.com/widgets.js" charset="utf-8"></script>&ldquo;Using crowd innovation to generate clinically-relevant AI algorithms will allow sharing of scarce human expertise to under-resourced healthcare settings to improve the quality of cancer care globally,&rdquo; the researchers wrote about the findings.&ldquo;These AI algorithms could improve cancer care globally by transferring the skills of expert clinicians to under-resourced healthcare settings,&rdquo; the authors concluded.<blockquote class="twitter-tweet" data-width="500" data-dnt="true">
What happens when <a href="https://twitter.com/hashtag/crowdinnovation?src=hash&amp;ref_src=twsrc%5Etfw">#crowdinnovation</a> contest draws coders from around the world to produce <a href="https://twitter.com/hashtag/AI?src=hash&amp;ref_src=twsrc%5Etfw">#AI</a> algorithms for <a href="https://twitter.com/hashtag/lungcancer?src=hash&amp;ref_src=twsrc%5Etfw">#lungcancer</a> tumor segmentation? Harvard Catalyst's Eva Guinan, senior author on <a href="https://twitter.com/JAMAOnc?ref_src=twsrc%5Etfw">@JAMAOnc</a> paper. <a href="https://twitter.com/ncats_nih_gov?ref_src=twsrc%5Etfw">@ncats_nih_gov</a> <a href="https://twitter.com/LISHarvard?ref_src=twsrc%5Etfw">@LISHarvard</a> <a href="https://twitter.com/klakhani?ref_src=twsrc%5Etfw">@klakhani</a> <a href="https://twitter.com/hashtag/CTSAProgram?src=hash&amp;ref_src=twsrc%5Etfw">#CTSAProgram</a> <a href="https://t.co/MkbZli6lYL">https://t.co/MkbZli6lYL</a>
&mdash; Harvard Catalyst (@HarvardCatalyst) <a href="https://twitter.com/HarvardCatalyst/status/1118947095560507392?ref_src=twsrc%5Etfw">April 18, 2019</a>
</blockquote><script async src="https://platform.twitter.com/widgets.js" charset="utf-8"></script>https://twitter.com/mraphaelmd/status/1119003385745817600

Jordana Jampel

Positive Preliminary Results From Phase 1 Trial of Dopaminergic Neuron Graft for Parkinson’s Disease

Dementia

Bradley Warady, MD

Joel M. Topf, MD, FACP

2025 Hume Award Winner Is Dedicated to Children’s Care

NKF Spring Clinical Meetings 2025

Distant Brain Metastases in Cutaneous Melanoma May Be Mitigated With Multiple Pathway Inhibitors

Melanoma

Latest News

Crowd Sourced AI Can Detect Cancer in Patients

A combined approach of crowd innovation and artificial intelligence (AI) algorithms can rapidly produce

About Us

Editorial Policy

Advertising

Contact Us

eNewsletters

Contribute

Career Center

MashupMD

Blood Cancers Today

Cancer Nursing Today

GU Oncology Now

Urban Health Today

Bile Duct Cancer

Colorectal Cancer

Gastroesophageal Cancer

Gastric Cancer

GEP-NETs

GIST

Liver Cancer

Pancreatic Cancer

GI Oncology Now

Anemia

Deep Vein Thrombosis

Essential Thrombocythemia

Hemophilia

Immune Thrombocytopenic Purpura

Myelodysplastic Syndromes

Myelofibrosis

Myeloproliferative Neoplasms

Polycythemia Vera

Pulmonary Embolism

Sickle Cell Disease

Thrombocytopenia

Thrombophilia

Thrombosis

Thrombotic Thrombocytopenic Purpura

Venous Thromboembolism

von Willebrand Disease

Heme Today

Acute Kidney Injury

ADPKD

Anemia and Kidney Disease

Chronic Kidney Disease

COVID-19 and Kidney Disease

Diabetes and Hypertension

End-Stage Renal Disease

Gout

Hyperkalemia

IgAN

Kidney Transplantation

Metabolic Acidosis

Nephrology Times

HER2 Breast Cancer

HR Breast Cancer

Triple-Negative Breast Cancer

Genetics & Breast Cancer

Breast Cancers Today

Atherosclerotic Disease

Atrial Fibrillation

Heart Failure

Hypertension

Interventional Cardiology

Lipid Management

Practice Tips With Dr. Lichaa

Preventive Cardiology

Stroke

Cardiology

Breast Cancer

Gynecologic Cancer

Head and Neck Cancer

Skin Cancer

Oncology

Sleep

Urology

Diabetes

Ophthalmology

— Demodex Blepharitis

Neurology

Rheumatology

Gastroenterology

Pulmonology

Infectious Diseases

— HIV

Health and Wellness

Lupus

Future of Medicine

Orthopedics

Pharmacology

Career News

More

Conferences

CardioNerds

The Sleep Source With Dr. Morse

Rheumatology Rounds With Dr. Maheswaranathan

Partners

Expert Interviews

Subscribe