Skip to main content


Artificial Intelligence Speeds Brain Tumor Diagnosis

Posted on by

Real time diagnostics in the operating room
Caption: Artificial intelligence speeds diagnosis of brain tumors. Top, doctor reviews digitized tumor specimen in operating room; left, the AI program predicts diagnosis; right, surgeons review results in near real-time.
Credit: Joe Hallisy, Michigan Medicine, Ann Arbor

Computers are now being trained to “see” the patterns of disease often hidden in our cells and tissues. Now comes word of yet another remarkable use of computer-generated artificial intelligence (AI): swiftly providing neurosurgeons with valuable, real-time information about what type of brain tumor is present, while the patient is still on the operating table.

This latest advance comes from an NIH-funded clinical trial of 278 patients undergoing brain surgery. The researchers found they could take a small tumor biopsy during surgery, feed it into a trained computer in the operating room, and receive a diagnosis that rivals the accuracy of an expert pathologist.

Traditionally, sending out a biopsy to an expert pathologist and getting back a diagnosis optimally takes about 40 minutes. But the computer can do it in the operating room on average in under 3 minutes. The time saved helps to inform surgeons how to proceed with their delicate surgery and make immediate and potentially life-saving treatment decisions to assist their patients.

As reported in Nature Medicine, researchers led by Daniel Orringer, NYU Langone Health, New York, and Todd Hollon, University of Michigan, Ann Arbor, took advantage of AI and another technological advance called stimulated Raman histology (SRH). The latter is an emerging clinical imaging technique that makes it possible to generate detailed images of a tissue sample without the usual processing steps.

The SRH technique starts off by bouncing laser light rapidly through a tissue sample. This light enables a nearby fiberoptic microscope to capture the cellular and structural details within the sample. Remarkably, it does so by picking up on subtle differences in the way lipids, proteins, and nucleic acids vibrate when exposed to the light.

Then, using a virtual coloring program, the microscope quickly pieces together and colors in the fine structural details, pixel by pixel. The result: a high-resolution, detailed image that you might expect from a pathology lab, minus the staining of cells, mounting of slides, and the other time-consuming processing procedures.

To interpret the SRH images, the researchers turned to computers and machine learning. To teach a computer, it must be fed large datasets of examples in order to learn how to perform a given task. In this case, they used a special class of machine learning called deep neural networks, or deep learning. It’s inspired by the way neural networks in the human brain process information.

In deep learning, computers look for patterns in large collections of data. As they begin to recognize complex relationships, some connections in the network are strengthened while others are weakened. The finished network is typically composed of multiple information-processing layers, which operate on the data to return a result, in this case a brain tumor diagnosis.

The team trained the computer to classify tissues samples into one of 13 categories commonly found in a brain tumor sample. Those categories included the most common brain tumors: malignant glioma, lymphoma, metastatic tumors, and meningioma. The training was based on more than 2.5 million labeled images representing samples from 415 patients.

Next, they put the machine to the test. The researchers split each of 278 brain tissue samples into two specimens. One was sent to a conventional pathology lab for prepping and diagnosis. The other was imaged with SRH, and then the trained machine made a diagnosis.

Overall, the machine’s performance was quite impressive, returning the right answer about 95 percent of the time. That’s compared to an accuracy of 94 percent for conventional pathology.

Interestingly, the machine made a correct diagnosis in all 17 cases that a pathologist got wrong. Likewise, the pathologist got the right answer in all 14 cases in which the machine slipped up.

The findings show that the combination of SRH and AI can be used to make real-time predictions of a patient’s brain tumor diagnosis to inform surgical decision-making. That may be especially important in places where expert neuropathologists are hard to find.

Ultimately, the researchers suggest that AI may yield even more useful information about a tumor’s underlying molecular alterations, adding ever greater precision to the diagnosis. Similar approaches are also likely to work in supplying timely information to surgeons operating on patients with other cancers too, including cancers of the skin and breast. The research team has made a brief video to give you a more detailed look at the new automated tissue-to-diagnosis pipeline.


[1] Near real-time intraoperative brain tumor diagnosis using stimulated Raman histology and deep neural networks. Hollon TC, Pandian B, Adapa AR, Urias E, Save AV, Khalsa SSS, Eichberg DG, D’Amico RS, Farooq ZU, Lewis S, Petridis PD, Marie T, Shah AH, Garton HJL, Maher CO, Heth JA, McKean EL, Sullivan SE, Hervey-Jumper SL, Patil PG, Thompson BG, Sagher O, McKhann GM 2nd, Komotar RJ, Ivan ME, Snuderl M, Otten ML, Johnson TD, Sisti MB, Bruce JN, Muraszko KM, Trautman J, Freudiger CW, Canoll P, Lee H, Camelo-Piragua S, Orringer DA. Nat Med. 2020 Jan 6.


Video: Artificial Intelligence: Collecting Data to Maximize Potential (NIH)

New Imaging Technique Allows Quick, Automated Analysis of Brain Tumor Tissue During Surgery (National Institute of Biomedical Imaging and Bioengineering/NIH)

Daniel Orringer (NYU Langone, Perlmutter Cancer Center, New York City)

Todd Hollon (University of Michigan, Ann Arbor)

NIH Support: National Cancer Institute; National Institute of Biomedical Imaging and Bioengineering

A Scientist Who Bends Musical Notes

Posted on by

As a pioneer in cancer immunotherapy, Jim Allison has spent decades tackling major scientific challenges. So it’s interesting that Allison would consider one of the top five moments in his life jamming onstage with country star Willie Nelson. Yes, in addition to being a top-flight scientist at the University of Texas MD Anderson Cancer Center, Houston, Allison plays a mean harmonica.

Allison taught himself how to bend notes on the harmonica as a teenager growing up in a small Texas town. By his 20s, Allison was good enough to jam a couple of nights a week with the now legendary Clay Blaker & the Texas Honky Tonk Band. When Blaker asked if he wanted to hit the road with the band, Allison declined. He had his postdoctoral training to finish in molecular immunology.

Optimizing Radio-Immunotherapy for Cancer

Posted on by

Zachary Morris

Zachary Morris
Credit: Alan Leon

Zachary Morris has certainly done some memorable things. As a Rhodes Scholar, he once attended an evening reception at Buckingham Palace, played a game of pick-up football with former President Bill Clinton, and traveled to South Africa to take a Robben Island Prison tour, led by the late Nelson Mandela. But something the young radiation oncologist did during his medical residency could prove even more momentous. He received a special opportunity from the American Board of Radiology to join others in studying how to pair radiation therapy with the emerging cancer treatment strategy of immunotherapy.

Morris’s studies in animals showed that the two treatments have a unique synergy, generating a sustained tumor-specific immune response that’s more potent than either therapy alone. But getting this combination therapy just right to optimize its cancer-fighting abilities remains complicated. Morris, now a researcher and clinician at the University of Wisconsin School of Medicine and Public Health, Madison, has received a 2017 NIH Director’s Early Independence Award to look deeper into this promising approach. He and his collaborators will use what they learn to better inform their future early stage clinical trials of radio-immunotherapy starting with melanoma, head and neck cancers, and neuroblastoma.

Summer Reading Suggestions from Scientists: Robert Horvitz

Posted on by

Summer ReadingTwo Science Selections: 

Horace Freeland Judson, Eighth Day of Creation. A comprehensive history of the origins and early science of the field of modern molecular biology, written by historian Horace Freeland Judson based on personal interviews with those who drove the revolution in biology. First and foremost are the science—DNA, RNA and protein, the genetic code, and gene regulation—and the scientific process—the seed ideas, the “aha” insights and the brilliant and elegant experiments. But this book is also the story of scientists in the process of discovery and of how the science that emerged was at least as much a consequence of the personalities as of the experimental skills of those involved. Fascinating, engaging, and fun—I’ve recommended this book to many, scientist and non-scientist alike.

Georgina Ferry, Dorothy Hodgkin. A superb biography of one of modern science’s most exceptional and distinguished pioneers. Awarded the Nobel Prize in Chemistry in 1964 for determining the crystal structures of penicillin and vitamin B12, Dorothy Crowfoot Hodgkin faced repeated challenges as a woman attempting to study and then pursue a career in chemistry in the 1930s and 1940s in England. Hodgkin is only one of four women ever awarded the Nobel Prize in Chemistry; the others were Marie Curie (1911); her daughter Irene Joliot-Curie (1935); and Ada Yonath (2009). Once recognized, Hodgkin worked hard to combat social inequalities and was president for more than a decade of Pugwash, an international organization founded by Bertrand Russell and dedicated to preventing war. Hodgkin has been a role model for many, although she disagreed rather strongly with the political views and actions of her most famous student, Margaret Thatcher.

Personal Connection: 

George Klein, The Atheist and the Holy City. This book was a gift to me from George Klein, a Hungarian-Swedish tumor biologist and virologist at the Karolinska Institute in Stockholm. George and his wife Eva are best known in biological circles for their pioneering discovery of the role of the Epstein-Barr virus in Burkitt’s lymphoma and other neoplasms. This book, one of many George has written, is a compilation of essays that focus on science, but incorporate history, religion and philosophy. Its sections are entitled “The Wisdom and Folly of Scientists,” “Journeys,” “Viruses and Cancer” and “The Human Condition,” and collectively touch upon topics as diverse as DNA hybridization, the discovery of Rous sarcoma virus, and the life cycle of Schistosoma mansoni, as well as the Nazi death camps, scientific creativity, and the conviction that God is an example of man’s wishful thinking. Thought-provoking and uplifting, this book is a story of science and much more. A must read for all.Line

Bob Horvitz

Robert Horvitz
Credit: Aynsley Floyd/ AP Images for HHMI

Robert Horvitz, Ph.D. is the David H. Koch Professor of Biology at the Massachusetts Institute of Technology, and a member of the MIT McGovern Institute for Brain Research and the MIT Koch Institute for Integrative Cancer Research. Dr. Horvitz is co-winner of the 2002 Nobel Prize in Physiology or Medicine for discoveries concerning genetic regulation of organ development and programmed cell death.

Creative Minds: Complex Solutions to Inflammation

Posted on by

Hao Wu

Hao Wu

For nearly 20 years, Hao Wu has studied innate immunity, our body’s first line of defense against infection. One of her research specialties is the challenging technique of X-ray crystallography, which she uses to capture the atomic structure of key molecules that drive an inflammatory response. But for this method to work, the proteins have to be coaxed to form regular crystals—and that has often proven to be prohibitively difficult. Wu, now at Boston Children’s Hospital and Harvard Medical School, can be relentless in her attempts to crystallize difficult molecular structures, and this quality has helped her make a number of important discoveries. Among them is the seminal finding that innate immune cells process and internalize signals to handle invading microbes much differently than previously thought.

Innate immune cells, which include macrophages and neutrophils, patrol the body non-specifically, keeping a look out for signs of anything unusual. Using protein receptors displayed on their surfaces, these cells can sense distinctive molecular patterns on microbes, prompting an immediate response at the site of infection.

Wu has shown that these cells form previously unknown protein complexes that mediate the immune response [1, 2]. She received an NIH Director’s 2015 Pioneer Award to help translate her expertise in the structural biology of these signaling complexes into the design of new kinds of anti-inflammatory treatments. This award helps exceptionally creative scientists to pioneer transformative approaches to major challenges in biomedical and behavioral research.