Artificial intelligence is 95 percent accurate at diagnosing real-time intraoperative brain tumors


A novel method of combining advanced optical imaging with an artificial intelligence algorithm produces accurate, real-time intraoperative diagnosis of brain tumors, a new study finds.

Published in Nature Medicine on January 6, the study examined the diagnostic accuracy of brain tumor image classification through machine learning, compared with the accuracy of pathologist interpretation of conventional histologic images.

The results for both methods were comparable: the AI-based diagnosis was 94.6% accurate, compared with 93.9% for the pathologist-based interpretation.

The imaging technique, stimulated Raman histology (SRH), reveals tumor infiltration in human tissue by collecting scattered laser light, illuminating essential features not typically seen in standard histologic images.

The microscopic images are then processed and analyzed with artificial intelligence, and in under two and a half minutes, surgeons are able to see a predicted brain tumor diagnosis. Using the same technology, after the resection, they are able to accurately detect and remove otherwise undetectable tumor.

“As surgeons, we’re limited to acting on what we can see; this technology allows us to see what would otherwise be invisible, to improve speed and accuracy in the OR, and reduce the risk of misdiagnosis,” says senior author Daniel A. Orringer, MD, associate professor of Neurosurgery at NYU Grossman School of Medicine, who helped develop SRH and co-led the study with colleagues at the University of Michigan. “With this imaging technology, cancer operations are safer and more effective than ever before.”

How the Study Was Conducted

To build the artificial intelligence tool used in the study, researchers trained a deep convolutional neural network (CNN) with more than 2.5 million samples from 415 patients to classify tissue into 13 histologic categories that represent the most common brain tumors, including malignant glioma, lymphoma, metastatic tumors, and meningioma.

In order to validate the CNN, researchers enrolled 278 patients undergoing brain tumor resection or epilepsy surgery at three university medical centers in the prospective clinical trial.

Brain tumor specimens were biopsied from patients, split intraoperatively into sister specimens, and randomly assigned to the control or experimental arm.

Specimens routed through the control arm–the current standard practice–were transported to a pathology laboratory and went through specimen processing, slide preparation by technicians, and interpretation by pathologists, a process which takes 20-30 minutes.

The experimental arm was performed intraoperatively, from image acquisition and processing to diagnostic prediction via CNN.

Notably, the diagnostic errors in the experimental group were unique from the errors in the control group, suggesting that a pathologist using the novel technique could achieve close to 100% accuracy.

The system’s precise diagnostic capacity could also be beneficial to centers that lack access to expert neuropathologists.

“SRH will revolutionize the field of neuropathology by improving decision-making during surgery and providing expert-level assessment in the hospitals where trained neuropathologists are not available,” says Matija Snuderl, MD, associate professor in the Department of Pathology at NYU Grossman School of Medicine and a co-author of the study.

NYU Langone’s Brain and Spine Tumor Center Offers Cutting-Edge Treatment

Dr. Orringer joined NYU Langone in August 2019, bringing with him the SRH technology he helped to develop. NYU Langone’s Brain and Spine Tumor Center is the first to offer this technique, using Invenio’s NIO Laser Imaging System, in the Northeast.

The newest addition to the center’s comprehensive suite of neurosurgical imaging technologies, SRH works in concert with intraoperative MRI and fluorescence-guided surgery to provide high-resolution precision guidance for NYU Langone’s world-class neurosurgeons.

This shows brain tumor samples

Stimulated Raman histologic images of diffuse astrocytoma (left) and meningioma (right). Image is credited to Daniel Orringer, NYU Langone Health.

“NYU Langone’s Department of Neurosurgery has long been a leader in bringing the most advanced treatment options to our patients,” says John G. Golfinos, MD, Joseph P. Ransohoff Professor of neurology and chair of the Department of Neurosurgery. “With the addition of Dr. Orringer’s expertise and this game-changing technology, we’re now even better equipped to provide safe surgeries and quality outcomes for the most complex brain tumor cases.”

The implementation of this new system is the most recent of NYU Langone’s efforts to integrate artificial intelligence in clinical practice to improve diagnostics of cancer.

Researchers and clinicians at NYU Langone’s Perlmutter Cancer Center have made recent strides in lung cancer, breast cancer, and brain tumor.

The fatality rate due to brain cancer is the highest in Asia [1]. Brain cancer develops in the brain or spinal cord [2]. The various symptoms of brain cancer include coordination issues, frequent headaches, mood swings, changes in speech, difficulty in concentration, seizures and memory loss. Brain cancer is a form of tumor which stays in the brain or central nervous system [2].

Brain tumors are categorized into various types based on their nature, origin, rate of growth and progression stage [3,4]. Brain tumors can be either benign or malignant. Benign brain tumor cells rarely invade neighboring healthy cells, have distinct borders and a slow progression rate (e.g., meningiomas, pituitary tumors and astrocytomas (WHO Grade-I)).

Malignant brain tumor cells (e.g., oligodendrogliomas, high-grade astrocytomas, etc) readily attack neighboring cells in the brain or spinal cord, have fuzzy borders and rapid progression rates. Brain tumors can be further classified into two types based on their origin: primary brain tumors and secondary brain tumors.

A primary tumor originates directly in the brain. If the tumor emerges in the brain due to cancer existing in some other body organ such as lungs, stomach etc., then it is known as a secondary brain tumor or metastasis.

Further, grading of brain tumors is done as per the rate of growth of cancerous cells, i.e., from low to high grade. WHO categorizes brain tumors into four grades (I, II, III and IV) as per the rate of growth [2,5,6,7,8,9] (discussed later).

Brain tumors are also characterized by their progression stages (Stage-0, 1, 2, 3 and 4). Stage-0 refers to cancerous tumor cells which are abnormal, but do not spread to nearby cells. Stages-1, 2 and 3 denote cells which are cancerous and spreading rapidly. Finally in Stage-4 the cancer spreads throughout the body. It is for sure that many lives could be saved if cancer were detected at an early stage through fast and cost-effective diagnosis techniques. However, it is very difficult to treat cancer at higher stages where survival rates are low.

Brain cancer diagnosis can be either invasive or non-invasive. Biopsy is the invasive approach where an incision is done to collect a tumor sample for examination.

It is considered the gold standard for cancer diagnosis where the pathologists observe various features of cells of the tumor sample under a microscope to confirm malignancy.

The physical examination of the body and brain scanning using imaging modalities constitute non-invasive approaches. The various imaging modalities such as computed tomography (CT), or magnetic resonance imaging (MRI) of brain are faster and safer techniques than biopsy.

These imaging modalities help radiologists locate brain disorders, observe disease progression and in surgical planning [10]. Brain scans or brain image reading to rectify disorders is however subject to inter-reader variability and accuracy which depends on the proficiency of the medical practitioner [11].

The advent of powerful computing machines and decreased hardware costs has led to the development of many computer-assisted tools (CAT) for cancer diagnosis by the research community. It is projected that CAT may help radiologists in improving the precision and consistency of the diagnostic results. In this study, various CAT-based intelligent learning methods i.e., machine learning (ML) and deep learning (DL) for automatic tissue characterization and tumor segmentation has been discussed. The basic objective of this paper is to highlight state-of-the-art of brain tumor classification methods, current achievements, challenges, and find the future scope.

Pathophysiology of Brain Cancer

The pathophysiology of brain cancer is discussed here. The reasons of occurrence of brain cancer are given from the perspective of cellular architecture and its functioning within the human body.

Cellular Level Architecture

The cell is the basic building block of the human body. It also defines the function of each organ within the body such as oxygen flow, blood flow and waste materials management. Each cell has a central control system known as the nucleus which contains 23 pairs of chromosomes consisting of millions of genes.

The instructions for these genes are contained within deoxyribonucleic acid (DNA) [12], which is like a blueprint for genes and defines their behavior. The protein of the gene is like a messenger that communicates between the cells or between the genes themselves. The message conveyed is defined by its 3D structure [13].

Genes control the continuous process of the death of unhealthy or unwanted cells besides reproduction of healthy cells. The main cause of a cancer is uncontrolled growth of cells. A mutation alters this DNA sequence, which is the root cause of malfunctioning of the genes. There are many factors involved in DNA mutations such as environmental, lifestyle, and eating habits.

The genes responsible for cancer are divided into three categories. We introduce and define each category in detail:

  • (i) The first category is known as tumor suppressors that controls the cell death cycle (apoptosis) [14]. This process has two signaling pathways. In the first pathway, the signal is generated by a cell to kill itself while in the second, the cell receives the death signal from nearby cells. This process of cell death is slowed down by a mutation in one of the pathways. It stops completely if this mutation happens in both pathways, leading to unstoppable cell growth [14,15]. Some examples of cell suppressor genes are RB1, PTEN, which are responsible for cell death [16].
  • (ii) The second category of genes is responsible for the repair of the DNA. Example of DNA repair genes are MGMT and p53 protein. Any malfunctioning in them may trigger cancer.
  • (iii) The third group known as proto-oncogenes, are in opposition to the function of the tumor suppressor genes and are responsible for the production of the protein fostering the division process and inhibiting the normal cell death [17,18]. In healthy cells, the cell division cycle is controlled by proto-oncogenes via protein signals which are generated by the cell itself or the connected cells. Once the signal is generated, it goes through a series of different steps, which is called signal transduction cascade or pathway as shown in Figure 1. This signal may be generated by the cell itself or from the nearby cells that are directly connected to it. In this pathway, many proteins are involved to carry the signal from the cell membrane to nucleus through the cytoplasm. In this process the cell membrane receptor accepts the signal and carries the message to nucleolus through various intermediate factors. Once, the signal reaches to the nucleus, the responsible genes for transcription is activated and performs the cell division task. One of the known proto-oncogenes responsible for the transcription is RAS which acts as a switch to turn ‘on’ or ‘off’ the cell division process [19]. Mutation alters its functionality which leads to transform this gene into an oncogene. In this situation the gene is unable to switch off the cell division signal and unstoppable growth of the cells may begin.
An external file that holds a picture, illustration, etc.
Object name is cancers-11-00111-g001.jpg
Figure 1
Cell cycle proliferation. (image courtesy: AtheroPointTM, Roseville, CA, USA).

If cancer starts in the body due to any of the above-mentioned reasons, it is known as a primary tumor which invades other organs directly. If the cancer starts through blood vessels then it known as secondary tumor or metastasis [20]. Even though the secondary tumor is formed, it needs oxygen, nutrients and a blood supply to survive. Many genes exist in the body to detect these needs and start establishing a vascular network for them to satisfy their needs. This process is known as angiogenesis and is another cause of cancer explosion [21]. The genes discussed above as well as their expended form has given in Table 1.

Table 1

Genomics relevance with Brain Tumor, RKT: Receptor Tyrosine Kinase, TP53 (p53): Tumor Protein53, RB1: Retino Blastoma1, EGFR: Epidermal Growth Factor Receptor, PTEN: Phosphatase and Tensin Homolog, IDH1/DH2: Isocitrate Dehydrogenase 1/2, 1p and 19 co-deletion, MGMT: O6-methylguanine DNA methyltransferase, BRAF: B-Raf proto-oncogene, ATRX: The α-thalassemia-mental retardation syndrome X-linked, HGG: High-Grade Gliomas, GBM: Glioblastoma.

Gene TypeFunctionMutation EffectRelevancy Between Brain Tumor and Genes [Degree of Mutation]
DNA repair
Initiating Apoptosis
Genetic InstabilityReduced ApoptosisAngiogenesisMore relevant to HGGBrain Tumor (80%)
Tumor SuppressorBlocks cell cycle progressionUnchecked cell cycle progressionMore relevant to GBMBrain Tumor (75%)
Trans-Membrane Receptor In (RTK)Increased ProliferationIncreased Tumor Cell SurvivalPrimary GBM (Approx. 40%)
Tumor SuppressorIncreased Cell ProliferationReduced Cell DeathPrimary GBM (15–40%)GBM (up to 80%)
IDH1 and DH2
Control citric acid cycleInhibits the function of enzymesIDH1
Primary GBM (5%)GBM Grade II-III (70–80%)IDH1 longer survival.
Relevant to oligodendroglial tumors
1p and 19q
Prognosis of the disease or treatment assessmentPoor prognosisOligodendrogliomas (80%)Anaplastic Oligodendrogliomas (60%)Oligoastrocytomas (30–50%)Anaplastic Oligoastrocytomas (20–30%)
DNA repair
predict patient survival
Cell proliferationGBM (35–75%)
Proto-oncogeneCell ProliferationApoptosisPilocyticastrocytomas (65–80%)Pleomorphic Xanthoastrocytomas and Gangliogliomas (25%)
Deposition of Genomic Repeats.Genital Anomalies,Hypotonia,Intellectual DisabilityMild-To-Moderate AnemiaSecondary To α-ThalassemiRelevent to oligodendroglial

About 15 percent of cancers worldwide are caused by viruses [22]. The viruses infect cells by altering DNA in the chromosomes which are responsible for converting proto-oncogenes into oncogenes. Only a few cancer causing viruses have been identified i.e., DNA virus and retroviruses or oncorna viruses (an RNA virus).

The four basic DNA viruses responsible for human cancers are human papillomavirus, Epstein-Barr, Hepatitis B and human herpes virus. The RNA viruses which cause cancer are Human T lymphotropic type1 and hepatitis C.

Several environmental factors also affect the cells. X-rays, UV light, viruses, tobacco products, pollution and many other daily use chemicals carry carcinogenic agents. Sunlight may also alter tumor suppressor genes in skin cells leading to skin cancer.

Further, the carcinogenic compounds in smoke alters the lung cells causing lung cancer [23].

Many studies have shown that tumor cells have unique molecular signatures and characteristics [24]. Hyperplasia, metaplasia, anaplasia, dysplasia, and neoplasia are the various stages of the cells that define the cellular abnormality during microscopic analysis.

Hyperplasia is the stage, where abnormal growth of the cell starts but the cell continues to appear normal. The cell first begins to appear abnormal in metaplasia.

In the anaplasia state, cells lose their morphological features and are difficult to discriminate. The cell appears to be abnormal and little aggressive in dysplasia.

Anaplasia is the most aggressive stage of this abnormal cell growth, where they seem quite abnormal and invade the surrounding tissues or start flowing through the bloodstream, which is one of the leading causes of metastasis [25].

The physical changes in cells due to cancer can be captured using high resolution imaging such as MRI or CT imaging, which are the focus of the next section.

Relevancy between Brain Tumor and Genes

As discussed in the last section, mutations in certain types of genes define the cancer. In various studies, some connection is found between degree of mutation in genes and type of brain tumor, which we have summarized in Table 1.

Tumor protein-53 (TP53) is involved in DNA repair and initiating apoptosis. Tp53 level is found to be quite abnormal in high-grade gliomas and mutations have been found in more than 80% of tumors [26]. The retinoblastoma (RB1) gene is a tumor suppression gene. RB1 mutation is found in approximately 75% of brain tumors and it is more relevant to glioblastoma [26]. EGFR is a trans-membrane receptor in the receptor tyrosine kinase (RTK) family. Mutation in EGFR will lead to increased cell cycle proliferation and increased tumor cell survival. It is generally associated with primary glioblastomas and approximately 40% of the mutations that caused them are found within it [27].

PTEN is a tumor suppressor gene and are responsible for about 15–40% of mutations found in primary glioblastomas. The degree of mutation may be up to 80%, indifferent glioblastoma [27]. IDH1 and IDH2 are enzymes that control the citric acid cycle. Mutations in them inhibit enzyme activity. Generally, IDH1 mutation is found less in primary glioblastoma patients (5%), but more in high grade glioblastomas (70–80%). IDH2 mutations are generally seen in oligodendroglial tumors [28].

Co-deletion of chromosomes 1p and 19q is indicative of oligodendroglial lineage and mainly seen in anaplastic oligoastrocytomas (20–30%), oligoastrocytomas (30–50%), anaplastic oligodendrogliomas (60%) and oligodendrogliomas (80%). 1p/19q helps in prognosis and treatment assessment [29]. MGMT protein is another DNA repair gene, for which 35–75% abnormality is found in glioblastomas [30].

BRAF is a proto-oncogene encoded as BRAF protein, which is involved in the cell proliferation cycle, apoptosis process and treatment assessment. BRAF mutations are generally found in pilocyticastrocytomas (65–80%), pleomorphic xanthoastrocytomas (about 80%) and gangliogliomas (25%) [26].

A-Thalassemia-mental retardation syndrome X-linked (ATRX) is a gene that encodes a protein and is associated with TP53 and IDH1 mutations. It is use as a prognostic indicator when tumors have anIDH1 mutation and it distinguishes between the tumors of oligodendroglial origin [26].

NYU Langone Health
Media Contacts:
Annie Harris – NYU Langone Health

Original Research: Open access
“Near real-time intraoperative brain tumor diagnosis using stimulated Raman histology and deep neural networks”. Daniel Orringer et al.
Nature Medicine doi:10.1038/s41591-019-0715-9.


Please enter your comment!
Please enter your name here

Questo sito usa Akismet per ridurre lo spam. Scopri come i tuoi dati vengono elaborati.