brain tumor detection using deep learning research paper

• Open access
• Published: 23 January 2023

MRI-based brain tumor detection using convolutional deep learning methods and chosen machine learning techniques

• Soheila Saeedi 1 ,
• Sorayya Rezayi 1 ,
• Hamidreza Keshavarz 2 &
• Sharareh R. Niakan Kalhori 1 , 3  

BMC Medical Informatics and Decision Making volume  23 , Article number:  16 ( 2023 ) Cite this article

32k Accesses

55 Citations

1 Altmetric

Metrics details

Detecting brain tumors in their early stages is crucial. Brain tumors are classified by biopsy, which can only be performed through definitive brain surgery. Computational intelligence-oriented techniques can help physicians identify and classify brain tumors. Herein, we proposed two deep learning methods and several machine learning approaches for diagnosing three types of tumor, i.e., glioma, meningioma, and pituitary gland tumors, as well as healthy brains without tumors, using magnetic resonance brain images to enable physicians to detect with high accuracy tumors in early stages.

Materials and Methods

A dataset containing 3264 Magnetic Resonance Imaging (MRI) brain images comprising images of glioma, meningioma, pituitary gland tumors, and healthy brains were used in this study. First, preprocessing and augmentation algorithms were applied to MRI brain images. Next, we developed a new 2D Convolutional Neural Network (CNN) and a convolutional auto-encoder network, both of which were already trained by our assigned hyperparameters. Then 2D CNN includes several convolution layers; all layers in this hierarchical network have a 2*2 kernel function. This network consists of eight convolutional and four pooling layers, and after all convolution layers, batch-normalization layers were applied. The modified auto-encoder network includes a convolutional auto-encoder network and a convolutional network for classification that uses the last output encoder layer of the first part. Furthermore, six machine-learning techniques that were applied to classify brain tumors were also compared in this study.

The training accuracy of the proposed 2D CNN and that of the proposed auto-encoder network were found to be 96.47% and 95.63%, respectively. The average recall values for the 2D CNN and auto-encoder networks were 95% and 94%, respectively. The areas under the ROC curve for both networks were 0.99 or 1. Among applied machine learning methods, Multilayer Perceptron (MLP) (28%) and K-Nearest Neighbors (KNN) (86%) achieved the lowest and highest accuracy rates, respectively. Statistical tests showed a significant difference between the means of the two methods developed in this study and several machine learning methods ( p -value < 0.05).

The present study shows that the proposed 2D CNN has optimal accuracy in classifying brain tumors. Comparing the performance of various CNNs and machine learning methods in diagnosing three types of brain tumors revealed that the 2D CNN achieved exemplary performance and optimal execution time without latency. This proposed network is less complex than the auto-encoder network and can be employed by radiologists and physicians in clinical systems for brain tumor detection.

Peer Review reports

Introduction

In medical terms, tumors are known as malignant or benign neoplasms, of which there are more than 200 diverse varieties that may affect humans [ 1 ]. According to the American Cancer Society, a brain tumor is a severe disease in which irregular brain tissue growth impairs brain function. The National Brain Tumor Foundation (NBTF) reported that the number of people who have lost their lives due to brain tumors has increased by 300% in the last three decades [ 2 ]. Brain tumors can lead to death if left untreated [ 3 ]. The complexity of brain tumors poses challenges for healthcare providers in diagnosing and caring for affected patients. Early detection of brain tumors and initiation of treatment play vital roles in the survival rate of these patients [ 4 ]. Brain tumor biopsy is not as easy as biopsy of other parts of the body, as it must be performed with surgery. Therefore, the need for another method for accurate diagnosis without surgery is crucial. Magnetic Resonance Imaging (MRI) is the best and most commonly used option for diagnosing brain tumors [ 5 ].

Recent advances in machine learning, particularly in deep learning, have led to the identification and classification of medical imaging patterns. Successes in this area include the possibility of retrieving and extracting knowledge from data instead of learning from experts or scientific texts. Machine learning is rapidly becoming a helpful tool for improving performance in various medical applications in various fields, including the prognosis and diagnosis of diseases, identification of molecular and cellular structures, tissue segmentation, and the classification of images [ 6 , 7 , 8 ]. In image processing, the most successful techniques currently used are Convolutional Neural Networks (CNNs), as they have many layers and high diagnostic accuracy if the number of input images is high [ 9 , 10 ]. Autoencoders are an unsupervised learning method in which neural networks are leveraged for representation learning. Remarkably, various deep learning and machine learning algorithms have been used to identify tumors (such as lung tumors) and detect cardiovascular stenosis. Moreover, performance evaluations have shown that they have high diagnostic accuracy [ 11 , 12 , 13 , 14 ].

Many studies have been conducted on the detection of brain tumors by various methods and models [ 5 , 15 , 16 , 17 , 18 , 19 , 20 , 21 ]. However, some of these studies have had a number of limitations, such as a lack of a performance comparison between the proposed model and traditional machine learning methods [ 5 , 22 , 23 ]. The proposed model in one study required complex computations [ 24 ]. The majority of relevant studies have provided models for classifying three types of brain tumors without including healthy subjects [ 22 , 23 , 24 , 25 ].

Speaking scientifically, tumor diagnosis by medical images is erroneous and depends heavily on the radiologist's experience. Because of widespread pathology variation and the possible fatigue of human specialists, researchers and physicians can benefit from computer-assisted interventions [ 6 ], and computational intelligence-oriented techniques can assist physicians in identifying and classifying brain tumors [ 5 ]. Machine learning approaches, especially deep learning, can also play a vital role in the analysis, segmentation, and classification of cancer images, especially brain tumors [ 26 ]. Furthermore, the use of such methods paves the way for accurate and error-free identification of tumors to recognize and distinguish them from other similar diseases. In the present study, we have tried to propose models that consider the suggestions and limitations presented in studies and suggest suitable solutions for them. Eight modeling methods have been compared to determine whether a significant difference exists between these methods in terms of performance.

Contributions of this work

The significant contributions of this work are detailed below:

Our networks are performed on an extensive dataset of 3264 T1-weighted contrast-enhanced MRI images, which are desirable for the training and testing phases.

The internal architecture of the modified 2D CNN and convolutional auto-encoder neural network are adjusted in terms of the number of layers, how the layers are positioned next to each other, the type of parameters and hyperparameters, and their values that can be varied to fine-tune our models to enhance accuracy.

Extracted essential features are utilized to classify three types of brain tumors and healthy brains (no tumor) by 2D CNN, auto-encoder network, and six common machine learning techniques.

In the modified 2D CNN, several convolution layers are considered; all layers in this hierarchical network have a 2*2 kernel function. This network consists of eight convolutional layers and four pooling layers; after all convolution layers, batch-normalization layers were applied. The training process was accomplished over 100 training epochs, and the batch size was 16. Each epoch last 7 s.

The auto-encoder network includes a convolutional auto-encoder network and a convolutional network for classification that uses the last output encoder layer of the first part. The encoder part has a convolutional layer of 32-filter length, two continuous convolutional layers with a filter length of 128, and two continuous convolutional layers with a filter length of 64. The decoding part of the network consists of a convolutional layer with a filter length of 32, two continuous convolutional layers with a filter length of 64, and two continuous convolutional layers with a filter length of 128 as well as a convolutional layer with a filter length of 128. For all convolution layers, the 2*2 kernel function was applied.

The developed networks achieved optimal accuracy of approximately 95% to 96%, and areas under the receiver operating characteristics curves (AUROC) are 0.99 or 1. Performance analysis proposes a renovation of our proposed techniques by comparing related papers.

One-way ANOVA for three parameters of precision, recall, and F-measure in eight modeling methods showed a statistically significant difference between the methods ( p -value < 0.001).

Our architectures attain competitive undertakings analogized with other state-of-the-art approaches on the MRI dataset and demonstrate a heightened generalization.

Related works

In recent years, many methodologies for classifying brain tumors by MRI images have been developed (Table 1 ).

A study conducted by Badža and Barjaktarovic´ in 2020 used a CNN to classify glioma, meningioma, and pituitary tumors. The network architecture applied in this study consisted of an input layer, two blocks “A,” two blocks “B,” a classification block, and an output layer, with 22 layers in total. Network performance was evaluated by employing the k-fold cross-validation method. The best value for the tenfold cross-validation method, which was obtained in this study, was 96.56%. The image dataset used in this study comprised 3064 T1-weighted contrast-enhanced MRI images from the Nanfang Hospital, General Hospital, and Tianjin Medical University in China [ 5 ].

In 2018 [ 24 ] developed capsule algorithms networks (DCNet) and diverse capsule networks (DCNet++). DCNet essentially adds a deeper convolutional network, leading to learning distinctive feature maps. DCNet++ uses a hierarchical architecture for learning, which makes it more efficient for learning complex data. They used a dataset comprising 3064 MRI images of 233 brain tumor patients for classification and considered only images of three types of brain tumors; a dataset of healthy people was not considered for classification. The DCNet model was developed by changing the eight initial convolutional layers to four layers with 16 kernels and was trained with eightfold cross-validation. The accuracy of the DCNet algorithm test was 93.04%, and the accuracy of the DCNet++ algorithm was 95.03%.

Gumaei et al. [ 23 ] introduced an automated approach to assist radiologists and physicians in identifying different types of brain tumors. The study was conducted in three steps: brain image preprocessing, brain feature extraction, and brain tumor classification. In the preprocessing step, brain images were converted into intensity brain images in the range of [0, 1], using a min–max normalization rule. In the next step, the PCA-NGIST method (a combination of normalized GIST descriptor with PCA) was adopted to extract features from MRI images. In the final step, Regularized Extreme Learning Machine (RELM) classification was applied to identify and classify the tumor types. The dataset provided by Cheng was used by the researchers in their study and consisted of 3064 MRI images from 233 patients divided into two subsets, 70% was used for training and 30% for classifier testing; a fivefold cross-validation method was utilized. The results reported 94.23% accuracy. The study, however, performed no comparative evaluation with other techniques, which can be considered as a study limitation [ 23 ].

Pashaei et al. [ 27 ] developed different methods to identify meningioma, glioma, and pituitary tumors. In their model, a CNN was used to extract hidden features from images and select features. The proposed model consisted of four convolutional layers, four pooling layers, one fully connected layer, and four batch normalization layers. The authors used ten epochs, 16 iterations per epoch, and the learning rate in this model was 0.01. The dataset provided by Cheng was also used in this study. The performance of the proposed model was evaluated using a tenfold cross-validation method, and 70% and 30% of the data was applied for training and system testing, respectively. The study compared the proposed method with MLP, Stacking, XGBoost, SVM, and RBF, and the results showed the high accuracy of the proposed method (93.68%) [ 27 ].

A CNN was also used by Abiwinanda in 2018 to diagnose the three most common types of brain tumors. In the learning process, the “adam” optimizer was used, which is a method for stochastic optimization using the stochastic gradient descent principle. In the study, the CNN was trained by 3064 T-1 weighted CE-MRI from brain tumor images provided by Cheng. The dataset included 1426 images of meningiomas, 708 images of gliomas, and 930 images of pituitary tumors. Of all the available images, 700 images from each class were applied, of which 500 were used for the training phase, and another 200 images were considered for the validation phase. In this model, all convolutional layers in the architectures used 32 filters, ReLu was used as an activation function, the maxpool kernel size was 2 × 2, and all the fully connected layers used 64 neurons. There were three neurons in the output layer, and the softmax activation function was employed at the output layer. The best-reported accuracy rates for training and validation were 98.51% and 84.19%, respectively [ 28 ].

In another study (2018), CNNs were applied to diagnose brain tumors using magnetic resonance images automatically. This study aimed to differentiate between healthy brains and brain tumor images. A two-stage multi-model system made the diagnosis. In the first stage, preprocessing and feature selection were performed by a CNN, and in the second stage, classification was done by an Error-Correcting Output Codes Support Vector Machine (ECOC-SVM). In the first stage, three algorithms, namely AlexNet, VGG-16, and VGG-19, were employed, among which AlexNet had the best performance with 99.55% accuracy. BraTS (2013 dataset) was used for the brain tumor localization phase, and images extracted from the standard Reference Image Database to Evaluate Response (RIDER) neuro MRI database were used for performance evaluation in the first phase [ 15 ].

Rehman et al. [ 22 ] studied three CNNs, namely AlexNet, GoogLeNet, and VGGNet. The study's primary purpose was to differentiate three brain tumor types, meningioma, glioma, and pituitary, using deep learning techniques and MRI images processing. Automated features were classified in the last phase using a linear classifier. Data augmentation techniques were applied to increase the sample size and reduce the possibility of over-fitting. The evaluation results showed that the VGG16 technique had the highest accuracy (98.69%) compared to other methods [ 22 ].

Mittal et al. [ 29 ] used the combination of Stationary Wavelet Transform (SWT) and a new Growing CNN (GCNN) to automate the segmentation process. In fact, they utilized these effective methods to identify brain tumors by MRI images. The evaluation results showed that the technique proposed in the study had the highest accuracy compared to the genetic algorithm; K-NN, SVM, and CNN [ 29 , 30 ].

Paul et al. [ 25 ] used deep learning methods to classify brain images related to meningioma, glioma, and pituitary tumors. In this research, the same dataset, i.e., 3064 T1-weighted contrast-enhanced MRI brain images of 233 patients, was applied; two types of neural networks, i.e., fully connected and CNNs, were designed. Moreover, a fivefold cross-validation technique showed that the general methods, with an accuracy of 91.43%, worked better than the specific methods, which required image dilation [ 25 ].

Material and methods

The methodology of the present study is illustrated in Fig.  1 . Major steps in the present study comprise brain tumor dataset selection, pre-processing MRI images, feature extraction, and classification by various classifiers.

Stages of the proposed methodology

The applied image-based dataset comprised 3264 T1-weighted contrast-enhanced MRI images [ 31 ]. There were four types of images in this dataset: glioma (926 images), meningioma (937 images), pituitary gland tumor (901 images), and healthy brain (500 images). All images were in sagittal, axial, and coronal planes. Figure  2 presents examples of the various types of tumors and different planes. The segment of tumors has been branded with a red outline. The number of images is different for each patient.

Description of normalized MRI images presenting diverse varieties of tumor in a different plane

Data augmentation and image pre-processing

Magnetic resonance images from this dataset had distinct sizes. These images represented the networks' input layer, so they were resized to 80*80 pixels. Each image was converted in two directions to augment the dataset. The first change included image rotation by 90°, and the second was flipping images vertically. Our chosen dataset was augmented three times, which resulted in 9792 images.

Proposed solutions

Figure  3 shows the proposed architecture for the two-dimensional CNN. A set of 9792 data was used in this study, 90% (8812) of which was employed as the training data and 10% (980) as the testing data. The proposed network had several layers, including convolution, which possessed two convolutional layers with 64 filters. Moreover, two convolution layers included 32 filters, and the others have 16. The final two convolutional layers make the desired network filters with a length of 8. The layers in this network have a 2*2 kernel function.

The architecture of the 2D convolution network

The convolutional network, which is also referred to as a neural network, has a hierarchical structure. This network creates a link between convolution layers, alternate pooling layers, and fully connected layers. One factor that should be noted here is that there is no need to use a pooling layer after each convolution layer. Figure  3 shows that the network has eight convolutional and four pooling layers. The final pooling layer with 2D output is changed to a 1D layer by flattened layers so it can be sent to the fully connected layers. Also, a type of padding is needed to manage and control the convolutional layer's output size. This study showed that the padding in adjacent cells is used for all networks to manage the edges of input data with the same values. To classify the data into categories by softmax activation function, a total of 1024 fully connected layer and a 4 fully connected layer were used. In this process, the batch-normalization layers were used to prevent overfitting. A dropout layer with a rate of 0.1 was also used following the max-pooling and fully connected layers.

For the activation function, the ReLU function was used in all layers apart from the last fully connected layer. To increase the efficiency, the Adam was used as an optimizing function. Different values, including 0.01, 0.001 and 0.0001, were used to test the learning rate parameter. Also, the best value with minimum learning error was found to be 0.001.

After 100 epochs, the training process was confirmed. The batch size was determined to be 16, and each epoch lasted about 7 s. The features extracted from the convolutional layer included input from the first layer fully connected to Ufc = 1024 hidden layers. The number of weights (Wconv) depended on the output size of the prior convolution layer (y1*y2), the number of filters (k), and the number of hidden layers in fully connected layers. Thus, the convolutional layer's weight was determined as follows [ 32 ]: Wconv = y1*y2*k*Ufc = 5*5*8*1024 = 204,800, where the number of existing parameters to the first fully connected layer equals 204,800 + 1024 (biases) = 205,824.

A summary of learning parameters for the proposed network can be seen in Table 2 . As seen in this table, the value of all parameters used to determine the four categories of this network are calculated by summing up the values in cited in the param column in Table 2 . The consequent value is 243,924, where all parameters are trainable.

Convolutional auto-encoder neural network

This study was conducted to design the architecture of a convolutional auto-encoder network. In this network, in order to predict the target value (Y) for the input (X), an auto-encoder was trained to predict the input (X) rather than training the network. The auto-encoder network was used to train and classify the data set instead of creating input images. Figure  4 shows the designed architecture of the convolutional auto-encoder network.

Convolutional auto-encoder network classification part

The network architecture designed in this study had two main parts. The first part included the convolutional auto-encoder network for data training, and the second part contained a convolutional network for classification, which utilizes the last output encoder layer of the first part. The first part of the architecture also consisted of 2D multilayer convolutional networks for both the encoder and the decoder. A total data set of 9792 was used in this study, 90% (8812) of which was used as the training data and 10% (980) as the test data.

The encoder part included a convolutional layer with a 32-filter length, two continuous convolutional layers with a 128-filter length, and two continuous convolutional layers with a 64-filter length. In the encoder, no pooling layer existed after each convolutional layer, but a second stage 2*2 max-pooling layer was considered after a sequence of two convolutional layers. The network’s decoder also included a convolutional layer with a 32-filter length, two continuous convolutional layers with a 64-filter length, two continuous convolutional layers with a 128-filter length, and a convolutional layer with a 128-filter length. A 2*2 kernel function was used for all convolutional layers, and there was no up-sampling layer after each convolutional layer. However, after a sequence of two convolutional layers, a 2*2 up-sampling layer was applied. The same padding was used for this network, and a batch normalization layer after each convolutional layer was considered. A dropout of 0.1 was also operated after each max-pooling layer, apart from the last layer, to prevent overfitting. Values of 0.01, 0.001 and 0.0001 were used to examine the learning rates, and the best value with minimum learning error was found to be 0. 001. The designed network was trained after 100 training epochs, and data was transmitted to the network in batches of 16 (batch-size), while each epoch ran in 14 s.

The critical features of the input data were removed by the automatic encoder network, and the output of the encoder layer was used for the classification (Fig.  5 ). For accurate classification, the output of the encoder layer was trained by two continuous convolution layers with 64-filter length, a 2*2-kernel function, and a 2*2-max-pooling layer with step 2. Batch-normalization and 0.1 dropout layers were also used to prevent overfitting [ 33 ]. To forward the output of the max-pooling layer to a 4-fully connected layer, the flattened layer was used [ 34 ], and the ReLU activation function was used for all layers.

Architecture of the proposed convolutional auto-encoder network

The vital factors for training and classification in the auto-encoder convolutional network include the encoder and classifier parameters. The extracted features of the encoder's last layer are trained by several convolutional layers, and the final extracted features would turn into the input of the first layer fully connected to the hidden layer of Ufc = 4. The number of weights (Wconv) depends on the number of hidden layers in the fully connected layer and the output size of the flattened layer. The flattened layer's output equals 5*5*64 = 1,600. Hence, the number of weights equals Wconv = out-flatten*Ufc = 1600*4 = 6,400, and the number of existing parameters to the second fully connected layer equals 6400 + 4 (biases) = 6404.

The learning parameters of this network are presented in Table 3 . The value of all modified parameters can be calculated by summing up the values in the param column (Table 3 ). The value of all modified parameters is 158,760, of which 1,569,000 are related to learning, and 960 are related to non-learning parameters.

Whole process in the present study was carried out in Keras with the Tensorflow backend. The networks in this study were designed in the Python environment and then, ran by cross-library in the Google Collaboratory (Colab) environment. Colab supplies a platform for running Python codes, especially machine learning, deep learning, and data analysis. The details of Colab hardware technical characteristics are given in Table 4 .

Performance evaluation metrics

The main objective of the current study was to classify MRI images into glioma, meningioma, pituitary gland tumor, and healthy brain classes. Metrics for performance evaluation included accuracy, precision, recall, and F-measure.

Accuracy refers to the proximity of a measured value to a standard or actual value. In other words, it is the ability of the tool to measure the exact value, whose accuracy can be measured.

In machine learning, precision results from dividing actual cases into sums of true and false cases. Recall is also the result of dividing the true items by all the items in that class. The weighting value for F-measure can be computed based on the precision and recall measures. F-measure is a good measure in evaluating the quality of classification and describing the weighted average between the quantities of precision and recall. The value of this measure is between 0 and 1, with 0 being the worst circumstance and 1 the best condition. This parameter was calculated by the following Eq. ( 4 ):

For organizing and evaluating classifiers and visualizing their performance, drawing receiver operating characteristics (ROC) plots can be useful in describing the results. ROC plots are commonly applied in medical decision-making and have recently been noticed in machine learning and data mining. The ROC curve is constructed by plotting the true positive rate (TPR) versus the false positive rate (FPR) in various threshold sets. Therefore, maximizing TPR while minimizing FPR are ideal achievements. This means that the upper left corner of the plot is the ideal point (FPR = 0 and TPR = 1).

Experimental results

Table 5 outlines the results of our proposed 2D CNN and convolutional auto-encoder neural network. The training accuracy of the proposed 2D CNN was found to be 96.4752%, whereas its validation accuracy was 93.4489%. The training accuracy of the proposed convolutional auto-encoder was found to be 95.6371%, and its validation accuracy was 90.9255%. The precision, recall, and F-measure of the four classes obtained from 2D CNN and the convolutional auto-encoder neural network are summarized in Tables 6 and 7 , respectively. Figure  6 shows the training, validation accuracy, and loss analyses of the proposed models concerning the number of epochs.

Training and validation analysis over 100 epochs for (1) 2D CNN: a training and testing accuracy analysis, and b training and testing loss analysis. (2) Convolutional auto-encoder neural network: c training and testing accuracy analysis, and d training and testing loss analysis

In the field of artificial intelligence, a confusion matrix is a matrix in which the performance of relevant algorithms is visualized. Each matrix column represents the predicted value of instances, and each row represents the actual (true) value of instances. This matrix justifies its appellation that allows us to see whether there are confusing results or overlaps between the classes. In medical research, it is significantly important to reduce the false positive and false negative outcomes in the modeling process. The impact of false positive and false negative rates is shown in Fig.  7 .

Confusion matrix analyses of the proposed model representing TP, TN, FP, and FN ratio obtained from the testing dataset of the a 2D CNN, and b convolutional auto-encoder neural network

Figure  8 presents the ROC curves of the proposed models along with classes 0, 1, 2, and 4 of the brain tumor classification models. The ideal point is observable for both class 0 and class 1.

Roc plots of the a 2D CNN, and b convolutional auto-encoder neural network

The outcomes of classical machine learning classifiers like Support Vector Machine (SVM), Logical Regression (LR), Random Forest (RF), Nearest Neighbor (NN), Stochastic Gradient Descent (SGD), and Multilayer Perceptron (MLP) were compared and classified into four classes. The obtained accuracy rates were 86% for NN, 82% for RF, 80% for SVM, 62% for LR, 52% for SGD, and 28% for MLP. Figure  9 shows the comparison of these results. The precision, recall, and F-measure for each set of glioma, meningioma, pituitary gland tumor, and healthy brain images were calculated by these methods and are summarized in Table 8 . For glioma tumor images, the highest precision was obtained by MLP (100%), the highest recall by KNN (90%), and the highest F-measure by KNN (87%). For meningioma tumors, the highest precision was obtained by KNN (93%), the highest recall by MLP (81%), and the highest F-measure by KNN (86%). For pituitary gland tumors, the highest precision was obtained by KNN (91%), the highest recall by RF (95%), and the highest F-measure by KNN (91%). For healthy brains, the highest precision was obtained by RF and SVM (83%), the highest recall by KNN (88%), and the highest F-measure by KNN (82%).

Comparison of classification accuracy rates of machine learning classifiers

The results of one-way ANOVA for the three parameters of precision, recall, and F-measure in eight modeling methods showed a statistically significant difference between the methods ( p -value < 0.001) (Table 9 ). LSD post hoc test results showed a significant difference between the means of precision, recall, and F-measure in the two methods presented in this study (2D CNN and convolutional auto-encoder) and the means of the three methods LR, SGD, and MLP ( p -value < 0.05). The mean F-measure parameter of the 2D CNN method, in addition to the three methods mentioned, was also significantly different from SVM ( p -value < 0.05) (Table 10 ).

The main objective of the current study was to develop two various deep learning networks and six machine learning techniques to classify MRI images into three classes of brain tumors (glioma, meningioma and pituitary gland tumor) and one class of healthy brain. The applied image dataset was publicly available at GitHub with 3264 T1-weighted contrast-enhanced magnetic resonance imaging (MRI) images.

According to the literature, some studies have used the famous T1-weighted contrast-enhanced MRI dataset (Figshare dataset), which contained 3064 MRI images of the human brain for tumor detection with computational approaches like neural networks. Studies using this dataset for the classification of brain tumors are listed in Table 11 . It should be noted that the study employed another dataset that included 3264 MRI images. This dataset contained four categories of MRI images, namely glioma, meningioma, and pituitary gland tumors and healthy brains (no tumors). Badža and Barjaktarović [ 5 ] conducted brain tumor detection using a CNN developed in MATLAB R2018a. Their proposed CNN had two convolutional layers of 64- and 16-filter lengths. The classification block had two fully connected layers: the first representing the flattened output of the max-pooling layer and the second having an equal number of hidden units to the number of tumor classes. The best result was reported as 95.40% for record-wise cross-validation for augmented images. Nonetheless, the highest accuracy obtained from the mentioned study (95.40%) with a value of 1.07 is less than our proposed 2D CNN. The execution time of our study was longer because of the complexity and high frequency of layers in the network, which justified the good accuracy we obtained. The longer execution time in the current study can be explained by the number of hidden layers, the pooling layers, and the batch sizes. It should be noted that the training of deeper networks requires extra time than the training of shallower or simpler networks [ 35 ].

In another research, a CNN and an extreme learning machine were applied to diagnose brain tumors. The proposed model utilized four convolution layers and batch-normalization layers with 16-, 32-, 64- and 128-filter lengths (3*3). Four ReLU layers and three max-pooling layers were used in the proposed CNN with stride size [ 2 , 2 ]. The model only had one fully connected layer with three types of classes. Feature vectors extracted by the mentioned convolution and layers were used as the input of KE-CNN (kernel CNN). The KE-CNN had 91.28% accuracy for classifying brain tumors [ 27 ]. However, our proposed 2D CNN and auto-encoder network achieved 96.47% and 95.63% accuracy, respectively. We used several layers of convolution for both networks and created complex networks, which can be justified by the large volume of data we used to increase classification accuracy. In comparison, other studies used networks with a small number of layers or a small amount of data [ 36 ].

In general, by comparing the two networks used in the current study, it can be concluded that the 2D CNN operated with 1% more accuracy than the auto-encoder network. Although the 2D CNN is more straightforward than the auto-encoder network, it performed better in feature extraction and learning, and according to what was previously mentioned, it uses all the parameters for learning [ 37 ]. The execution time (the duration of each epoch) or the runtime of the proposed convolutional network is less than that of the auto-encoder network. Therefore, the use of ordinary hardware and memory can be enough to run our proposed 2D CNN. One of the most notable differences between the current study and others is the use of six machine learning bribes to classify brain tumor images. SVM, NN, RF, SGD, LR, and MLP were developed for diagnosing brain tumors accurately.

In [ 23 ], researchers used a hybrid feature extraction approach with regularized extreme learning machine to classify types of brain tumors. Their method works by extracting the main features of brain images, and then applying principal component analysis to compute a covariance matrix of features. In the last step, a RELM is developed for diagnosing brain tumors into three classes (meningioma, pituitary, glioma). This method achieved 94.23% accuracy, which is not optimal compared to the results obtained from the networks designed in the current study.

A capsule network was employed in another study to classify brain tumors. Due to its good performance, the segmented tumor regions were applied as the inputs of the proposed capsule net. This method was implemented on Python 2.7, based on the Keras Library, using the Adam optimizer. Capsule net reached 86.56% accuracy for classifying segmented tumor regions and 78% accuracy for whole-brain tissue as input [ 21 ]. The researchers varied the feature maps in the convolutional layer of CapsNet in order to enhance accuracy; however, they achieved the highest accuracy of 86.56% using 64 feature maps with one convolutional layer of CapsNet. The network that employs only the tumor region or some other segmented part as input performs better in terms of execution speed. It also demands segmentation methods or a dedicated specialist to sign those parts [ 3 , 29 , 30 ]. The most favorable outcome in the research utilizing the segmented image parts as inputs has been presented by Tripathi and Bag [ 38 ], with 94.64% accuracy. They used features as inputs of classifiers extracted from the segmented brain tissue in the image. They checked their proposed method employing a fivefold cross-validation technique.

Like the approaches proposed in the current study, Rehman et al. performed the preprocessing of images with contrast improvement and dataset augmentation to reduce the occurrence of over-fitting and increase the database samples. Three types of CNNs (AlexNet, VGGNet, and GoogleNet) with an SVM model were utilized to diagnose brain tumors. The fine-tuned VGG16 network obtained the highest accuracy of 98.69% for the classification target [ 22 ]. In comparison, our developed methods 2% and 3% less accurate, respectively, than VGG16. This network is an intense and very deep network with 138 million weights, requiring complex hardware for calculating real-time performance [ 39 ]. Notably in this study, similar to ours, the researchers utilized various data augmentation methods to increase the size of the training dataset, such as rotating and flipping with raw MRI images. Data augmentation aims to enhance network performance by intentionally creating more training data from the original data.

In another study [ 28 ], CNNs were molded to determine the three most common types of brain tumors (i.e., glioma, meningioma, and pituitary gland tumors). In this study, researchers applied five different architectures of CNN for the classification of brain tumors and reported the highest accuracy for architecture 2. This architecture's training and validation accuracies were 98.51% and 84.19%, respectively. Architecture 2 is comprised of two convolutional layers, ReLU layer, and max-pooling with 64 hidden neurons. The testing accuracies of the developed networks in the current study were more significant than the accuracy of this architecture; nevertheless, this architecture has the capacity to overfit using a small learning rate or lower amount of training and testing data.

The current work is a pioneer study to develop two deep CNNs with optimal learning parameters and high accuracy. We also compared six machine learning techniques applied to classify brain tumors and healthy brains (no tumor). To better compare the previous studies conducted in this area, some key results are given in Table 11 .

A limitation of medical image classification is the small size of medical image databases. This limitation, in turn, restricts the availability of medical images for training deep neural networks. One way to deal with this challenge in our study is to apply data augmentation techniques to create new brain tumor lesions through scaling and rotation, which may cause class imbalance. In addition, in this study, our primary plan was to train the networks using local images of a hospital, but the problem of labeling the images prevented this from being implemented. Labeling cancer images is not only time-consuming but also requires a high level of expertise that is challenging in brain tumor analysis. In future works, considering the importance of rapid and accurate diagnosis of brain tumors without latency, we will investigate the constructions of other robust deep neural networks for brain tumor classification with less execution time and more simplicity. Hence, full machine learning and deep learning algorithms can be implemented as future enhancements. Furthermore, the proposed techniques can be used to detect different forms of cancers in MRI or Computed Tomography (CT) scan.

One of the areas of use for artificial intelligence and machine learning is the health domain. Deep networks are currently being designed and developed to detect diseases based on imaging. In order to do this, we have proposed computational-oriented methods to classify brain tumors. In our study, a novel 2D CNN architecture, a convolutional auto-encoder network, and six common machine-learning techniques were developed for brain tumor detection. This classification was conducted using a T1-weighted, contrast-enhanced MRI dataset, which includes three types of tumors and a healthy brain with no tumors.

According to the results and output shown in Figs. 6 , 7 and 8 , the proposed neural networks showed significant improvement over previous ones in detecting brain MRI image features and classifying them into three types of tumors and one class of healthy brain. The training accuracy of the proposed 2D CNN was found to be 96.47%, and the training accuracy of the proposed auto-encoder network was found to be 95.63%. In addition to the two-deep networks used in our study, six machine-learning techniques were also developed to classify brain tumors. The highest accuracies of 86%, 82% and 80% were attained for KNN, RF, and SVM, respectively. Comparing our networks with similar state-of-the-art methods shows that our proposed networks performed somewhat better with optimal execution time (maximum 15 min for 2D network and 25 min for auto-encoder network). The results of this study demonstrate that our proposed networks have an immeasurable generalization and high execution speed; therefore, they can be applied as effective decision-support agents for radiologists in medical diagnostics.

Availability of data and materials

All data generated or analyzed during this study are included in this published article. The link of the public MRI dataset that is used in this study is: https://www.kaggle.com/sartajbhuvaji/brain-tumor-classification-mri/ .

Prabukumar M, Agilandeeswari L, Ganesan K. An intelligent lung cancer diagnosis system using cuckoo search optimization and support vector machine classifier. J Ambient Intell Humaniz Comput. 2019;10(1):267–93.

Article   Google Scholar  

El-Dahshan ESA, Mohsen HM, Revett K, Salem ABM. Computer-aided diagnosis of human brain tumor through MRI: A survey and a new algorithm. Expert Syst Appl. 2014;41(11):5526–45.

Meng Y, Tang C, Yu J, Meng S, Zhang W. Exposure to lead increases the risk of meningioma and brain cancer: a meta-analysis. J Trace Elem Med Biol. 2020;60:126474.

Article   CAS   Google Scholar  

McFaline-Figueroa JR, Lee EQ. Brain tumors. Am J Med. 2018;131(8):874–82.

Badža MM, Barjaktarović MČ. Classification of brain tumors from MRI images using a convolutional neural network. Appl Sci. 2020;10(6):1999.

Shen D, Wu G, Suk H-I. Deep learning in medical image analysis. Annu Rev Biomed Eng. 2017;19(1):221–48.

Litjens G, Kooi T, Bejnordi BE, Setio AAA, Ciompi F, Ghafoorian M, et al. A survey on deep learning in medical image analysis. Med Image Anal. 2017;42:60–88.

Suzuki K. Overview of deep learning in medical imaging. Radiol Phys Technol. 2017;10(3):257–73.

Hijazi S, Kumar R, Rowen C. Using convolutional neural networks for image recognition. San Jose: Cadence Design Systems Inc; 2015. p. 1–12.

Google Scholar  

O’Shea K, Nash R. An introduction to convolutional neural networks. 2015. arXiv:1511.08458 .

Wong KKL, Fortino G, Abbott D. Deep learning-based cardiovascular image diagnosis: a promising challenge. Futur Gener Comput Syst. 2020;110:802–11.

Hua KL, Hsu CH, Hidayati SC, Cheng WH, Chen YJ. Computer-aided classification of lung nodules on computed tomography images via deep learning technique. OncoTargets Ther. 2015;8:2015–22.

CAS   Google Scholar  

Işın A, Direkoğlu C, Şah M. Review of MRI-based brain tumor image segmentation using deep learning methods. Proc Comput Sci. 2016;102:317–24.

Sarker P, Shuvo MMH, Hossain Z, Hasan S, editors. Segmentation and classification of lung tumor from 3D CT image using K-means clustering algorithm. In: 2017 4th international conference on advances in electrical engineering (ICAEE); 2017 28–30 Sept. 2017.

Abd-Ellah MK, Awad AI, Khalaf AAM, Hamed HFA. Two-phase multi-model automatic brain tumour diagnosis system from magnetic resonance images using convolutional neural networks. EURASIP J Image Video Process. 2018;2018(1):97.

Malathi M, Sinthia P. Brain tumour segmentation using convolutional neural network with tensor flow. Asian Pac J Cancer Prevent APJCP. 2019;20(7):2095.

Nalbalwar R, Majhi U, Patil R, Gonge S. Detection of brain tumor by using ANN. Image. 2014;2(3):7.

Amin J, Sharif M, Raza M, Yasmin M. Detection of brain tumor based on features fusion and machine learning. J Ambient Intell Humaniz Comput 2018;1–17.

Usman K, Rajpoot K. Brain tumor classification from multi-modality MRI using wavelets and machine learning. Pattern Anal Appl. 2017;20(3):871–81.

Farhi L, Zia R, Ali ZA. Performance analysis of machine learning classifiers for brain tumor MR images. Sir Syed Univ Res J Eng Technol. 2018;8(1):23–8.

Afshar P, Mohammadi A, Plataniotis KN. Brain tumor type classification via capsule networks. In: 2018 25th IEEE international conference on image processing (ICIP). IEEE; 2018 Oct 7. p. 3129–33.

Rehman A, Naz S, Razzak MI, Akram F, Imran M. A deep learning-based framework for automatic brain tumors classification using transfer learning. Circuits Syst Signal Process. 2020;39(2):757–75.

Gumaei A, Hassan MM, Hassan MR, Alelaiwi A, Fortino G. A Hybrid feature extraction method with regularized extreme learning machine for brain tumor classification. IEEE Access. 2019;7:36266–73.

Phaye SS, Sikka A, Dhall A, Bathula D. Dense and diverse capsule networks: making the capsules learn better. arXiv:1805.04001 . 2018 May 10.

Paul JS, Plassard AJ, Landman BA, Fabbri D. Deep learning for brain tumor classification. In: Paper presented at the Medical Imaging 2017: Biomedical Applications in Molecular, Structural, and Functional Imaging. 2017.

Jin KH, McCann MT, Froustey E, Unser M. Deep convolutional neural network for inverse problems in imaging. IEEE Trans Image Process. 2017;26(9):4509–22.

Pashaei A, Sajedi H, Jazayeri N. Brain tumor classification via convolutional neural network and extreme learning machines. In: 2018 8th international conference on computer and knowledge engineering (ICCKE). IEEE; 2018 Oct 25. p. 314–9.

Abiwinanda N, Hanif M, Hesaputra ST, Handayani A, Mengko TR. Brain tumor classification using convolutional neural network. In: World congress on medical physics and biomedical engineering 2018. Singapore: Springer; 2019. p. 183–9.

Mittal M, Goyal LM, Kaur S, Kaur I, Verma A, Jude HD. Deep learning based enhanced tumor segmentation approach for MR brain images. Appl Soft Comput. 2019;78:346–54.

Mohammed MA, Abd Ghani MK, Hamed RI, Ibrahim DA, Abdullah MK. Artificial neural networks for automatic segmentation and identification of nasopharyngeal carcinoma. J Comput Sci. 2017;21:263–74.

Lee JY, Lee KS, Seo BK, Cho KR, Woo OH, Song SE, et al. Radiomic machine learning for predicting prognostic biomarkers and molecular subtypes of breast cancer using tumor heterogeneity and angiogenesis properties on MRI. Eur Radiol. 2022;32(1):650–60.

Riazi R, Asrardel M, Shafaee M, Vakilipour S, Zare H, Veisi H. A data mining study on combustion dynamics and NOx emission of a swirl stabilised combustor with secondary fuel injection. Int J Heavy Veh Syst. 2017;24(3):215–38.

Fouladi S, Safaei AA, Mammone N, Ghaderi F, Ebadi M. Efficient deep neural networks for classification of Alzheimer’s disease and mild cognitive impairment from scalp EEG recordings. Cogn Comput. 2022; 1–22.

Fouladi S, Ebadi M, Safaei AA, Bajuri MY, Ahmadian A. Efficient deep neural networks for classification of COVID-19 based on CT images: virtualization via software defined radio. Comput Commun. 2021;176:234–48.

Wang Y, Wang L, Wang H, Li P. End-to-end image super-resolution via deep and shallow convolutional networks. IEEE Access. 2019;7:31959–70.

Hussain S, Anwar SM, Majid M. Segmentation of glioma tumors in brain using deep convolutional neural network. Neurocomputing. 2018;282:248–61.

Khalid S, Khalil T, Nasreen S. A survey of feature selection and feature extraction techniques in machine learning. In: 2014 science and information conference. IEEE; 2014 Aug 27. p. 372–8.

Tripathi PC, Bag S. Non-invasively grading of brain tumor through noise robust textural and intensity based features. In: Computational intelligence in pattern recognition. Springer; 2020. p. 531–9.

Han S, Pool J, Tran J, Dally W. Learning both weights and connections for efficient neural network. Adv Neural Inf Process Syst. 2015;28:1135–43.

Download references

Acknowledgements

Not applicable.

In this paper, we didn't have any financial sponsor.

Author information

Authors and affiliations.

Medical Informatics and Health Information Management Department, School of Allied Medical Sciences, Tehran University of Medical Sciences, 3rd Floor, No #17, Farredanesh Alley, Ghods St, Enghelab Ave, Tehran, 14177-44361, Iran

Soheila Saeedi, Sorayya Rezayi & Sharareh R. Niakan Kalhori

Faculty of Electrical and Computer Engineering, Tarbiat Modares University, Tehran, Iran

Hamidreza Keshavarz

Peter L. Reichertz Institute for Medical Informatics, TU Braunschweig and Hannover Medical School, 38106, Brunswick, Germany

Sharareh R. Niakan Kalhori

You can also search for this author in PubMed   Google Scholar

Contributions

SS, SR, HK, SRNK: Conceptualization, Methodology, Software, SS, SR, HK, SRNK: Data curation, Writing-Original draft preparation. SS, SR, HK, SRNK: Visualization, Investigation. HK, SRNK: Supervision. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Sorayya Rezayi .

Ethics declarations

Ethics approval and consent to participate.

All methods were carried out in accordance with relevant guidelines and regulations.

Consent for publication

Competing interests.

The authors declare no competing interests regarding the publication of this article.

Additional information

Publisher's note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Saeedi, S., Rezayi, S., Keshavarz, H. et al. MRI-based brain tumor detection using convolutional deep learning methods and chosen machine learning techniques. BMC Med Inform Decis Mak 23 , 16 (2023). https://doi.org/10.1186/s12911-023-02114-6

Download citation

Received : 04 June 2022

Accepted : 16 January 2023

Published : 23 January 2023

DOI : https://doi.org/10.1186/s12911-023-02114-6

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Convolutional neural network
• Brain tumor
• Machine learning
• Medical imaging

BMC Medical Informatics and Decision Making

ISSN: 1472-6947

• General enquiries: [email protected]

Brain Tumor Detection Using Machine Learning and Deep Learning: A Review

Affiliations.

• 1 Department of E & TC Engineering, College of Engineering, Pune, India.
• 2 Department of Electrical and Computer Engineering, Aarhus University, Aarhus, Denmark.
• PMID: 34561990
• DOI: 10.2174/1573405617666210923144739

According to the International Agency for Research on Cancer (IARC), the mortality rate due to brain tumors is 76%. It is required to detect the brain tumors as early as possible and to provide the patient with the required treatment to avoid any fatal situation. With the recent advancement in technology, it is possible to automatically detect the tumor from images such as Magnetic Resonance Iimaging (MRI) and computed tomography scans using a computer-aided design. Machine learning and deep learning techniques have gained significance among researchers in medical fields, especially Convolutional Neural Networks (CNN), due to their ability to analyze large amounts of complex image data and perform classification. The objective of this review article is to present an exhaustive study of techniques such as preprocessing, machine learning, and deep learning that have been adopted in the last 15 years and based on it to present a detailed comparative analysis. The challenges encountered by researchers in the past for tumor detection have been discussed along with the future scopes that can be taken by the researchers as the future work. Clinical challenges that are encountered have also been discussed, which are missing in existing review articles.

Keywords: Brain tumor; convolutional neural networks; deep learning; machine learning; magnetic resonance imaging; preprocessing.

Copyright© Bentham Science Publishers; For any queries, please email at [email protected].

Publication types

• Brain Neoplasms* / diagnostic imaging
• Deep Learning*
• Machine Learning
• Neural Networks, Computer

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• Healthcare (Basel)

A Deep Learning Approach for Brain Tumor Classification and Segmentation Using a Multiscale Convolutional Neural Network

Associated data.

The dataset used in the experiments is publicly available from Figshare ( http://dx.doi.org/10.6084/m9.figshare.1512427 (accessed on 31 January 2021)).

In this paper, we present a fully automatic brain tumor segmentation and classification model using a Deep Convolutional Neural Network that includes a multiscale approach. One of the differences of our proposal with respect to previous works is that input images are processed in three spatial scales along different processing pathways. This mechanism is inspired in the inherent operation of the Human Visual System. The proposed neural model can analyze MRI images containing three types of tumors: meningioma, glioma, and pituitary tumor, over sagittal, coronal, and axial views and does not need preprocessing of input images to remove skull or vertebral column parts in advance. The performance of our method on a publicly available MRI image dataset of 3064 slices from 233 patients is compared with previously classical machine learning and deep learning published methods. In the comparison, our method remarkably obtained a tumor classification accuracy of 0.973, higher than the other approaches using the same database.

1. Introduction

Automatic segmentation and classification of medical images play an important role in diagnostics, growth prediction, and treatment of brain tumors. An early tumor brain diagnosis implies a faster response in treatment, which helps to improve patients’ survival rate. Location and classification of brain tumors in large medical images databases, taken in routine clinical tasks by manual procedures, have a high cost both in effort and time. An automatic detection, location, and classification procedure is desirable and worthwhile [ 1 ].

There are several medical imaging techniques used to acquire information about tumors (tumor type, shape, size, location, etc.), which are needed for their diagnosis [ 2 ]. The most important techniques are Computed Tomography (CT), Single-Photon-Emission Computed Tomography (SPECT), Positron Emission Tomography (PET), Magnetic Resonance Spectroscopy (MRS), and Magnetic Resonance Imaging (MRI). These techniques can be combined to obtain more detailed information about tumors. Anyhow, MRI is the most used technique due to its advantageous characteristics. In MRI acquisition, the scan provides hundreds of 2D image slices with high soft tissue contrast using no ionizing radiation [ 2 ]. There are four MRI modalities used in diagnosis: T1-weighted MRI (T1), T2-weighted MRI (T2), T1-weighted contrast-enhanced MRI (T1-CE), and Fluid Attenuated Inversion Recovery (FLAIR). Each MRI modality produces images with different tissue contrasts; thus, some are more suited to search a specific kind of tissue than others. T1 modality is typically used to work with healthy tissues. T2 images are more appropriate to detect borders of edema regions. T1-CE images highlight tumor borders and FLAIR images favor the detection of edema regions in Cerebrospinal Fluid [ 3 ]. Provided that the goal of MRI image processing is to locate and classify brain tumors, T1-CE modality is adequate and, as it is shown in this paper, sufficient.

In the last decades, Brain Tumor Imaging (BTI) has grown at an exponential rate. More specifically, the number of works about brain tumor quantification based on MRI images has increased significantly [ 2 ]. Brain Tumor Segmentation (BTS) consists in differentiating tumor-infected tissues from healthy ones. In many BTS applications, the brain tumor image segmentation is achieved by classifying pixels, thus the segmentation problem turns into a classification [ 4 ].

The aim of the work presented in this paper is to develop and test a Deep Learning approach for brain tumor classification and segmentation using a Multiscale Convolutional Neural Network. To train and test the proposed neural model, a T1-CE MRI image dataset from 233 patients, including meningiomas, gliomas, and pituitary tumors in the common views (sagittal, coronal, and axial), has been used [ 5 ]. Figure 1 shows examples of these three types of tumors. Additional information on the dataset is included in Section 2.2 . Our model is able to segment and predict the pathological type of the three kinds of brain tumors, outperforming previous studies using the same dataset.

Examples of MRI images of the T1-CE MRI image dataset. Left: coronal view of a meningioma tumor. Center: Axial view of a glioma tumor. Right: sagittal view of a pituitary tumor. Tumor borders have been highlighted in red.

In the BTS field, two main tumor segmentation approaches can be found: generative and discriminative. Generative approaches use explicit anatomical models to obtain the segmentation, while discriminative methods learn image features and their relations using gold standard expert segmentations [ 2 ]. Published studies following the discriminative approach have evolved from using classical Machine Learning [ 6 , 7 , 8 , 9 ] to more recent Deep Learning techniques [ 10 , 11 , 12 , 13 , 14 ].

In works using classical Machine Learning techniques, the segmentation pipeline includes a preprocessing stage designed for feature extraction. For example, in Sachdeva et al. [ 6 ], a semi-automatic system obtained the tumor contour, and 71 features were then computed using the intensity profile, the co-occurrence matrix, and Gabor functions. In addition, skull stripping is a common preprocessing stage in classical discriminative approaches, although it presents drawbacks such as the selection of parameters or the need of prior information about the images and a high computational time [ 15 ]. The extracted features are used as the input for a classification or a segmentation stage. In the aforementioned work [ 6 ], two classifiers, SVM (Support Vector Machine) and ANN (Artificial Neural Network), were compared. The obtained accuracy ranged from 79.3% to 91.7% for SVM and from 75.6% to 94.9% for ANN. Analogously, Sharma and Chhabra [ 7 ] proposed an automatic brain tumor classification system using 10 features and a Back Propagation Network as the classifier, and they reached an accuracy of 95.3%. Iftekharuddin et al. [ 8 ] explored using fractal wavelet features as input to a SOM (Self Organizing Map) classifier and achieved an average precision of 90%. Other examples of tumor classification pipelines use instance-based learning: Havaei et al. [ 9 ] developed a semi-automatic system based on a kNN classifier. They used the well-known BRATS 2013 dataset and obtained the second-best score over the complete and core tests, reporting Dice similarities of 0.85 and 0.75 for complete and core tumor regions, respectively.

In the last years, the next step in the evolution of machine learning techniques has been to allow computers to discover the features that optimally represent the data. This concept is the foundation of Deep Neural Networks [ 4 ], which transform the problems to solve from feature-driven into data-driven. Within deep neural networks, Convolutional Neural Networks (CNN) [ 16 ] and Fully Convolutional Networks (FCN) [ 17 , 18 ] have been applied to a broad range of applications. In particular, they are widely used today for generic image processing [ 19 ] and specifically in medical image analysis [ 1 , 4 , 20 ].

The advent of Deep Learning has had a tremendous impact in the design of medical imaging systems, and newer techniques have been widely adopted by the researchers over the last few years [ 21 ]. The advances in the field have opened a very exciting time for collaboration of machine learning scientists and radiologists [ 22 ]. Havaei et al. [ 10 ] explored the use of a fully convolutional system for the BRATS 2013 dataset, just after their kNN classifier proposal [ 9 ], commented before. Most of the recent brain tumor classification studies using Deep Learning are based on CNNs or FCNs. Moeskops et al. [ 11 ] used a multiscale CNN to segment brain tissues and white matter hyperintensities. They used images from the MRBrainS13 challenge [ 12 ] to evaluate their CNN architecture and reached a precision of about 85%. One of the most significant advances in the field was achieved by Ronneberger et al. [ 13 ], who proposed an interesting CNN named U-net. They used this CNN architecture to segment neural structures in electron microscopy images from the ISBI challenge [ 14 ]. U-net is composed by 5 convolutional stages and 5 deconvolutional stages shaping a U, from where it takes its name. They obtained the best results in the ISBI challenge [ 14 ].

However, all the previously cited studies have in common that they aimed to segment areas belonging to brain tumors without the need to also classify these areas as belonging to different brain tumor types. As explained above, in our study the objective was to develop a new Deep Learning model for both tumor segmentation and classification. A relevant previous related work is the study of Mohsen et al. [ 23 ], who proposed a deep learning classifier combined with a discrete wavelet transform (DWT) and principal components analysis (PCA) to classify a dataset with three different brain tumors. Four other deep learning relevant studies following the same purpose have in common the use of the same dataset that we adopted in this work, which is important to contrast and endorse the performance results of the proposed model presented here. Abiwinanda et al. [ 24 ] proposed a two-layer CNN architecture. Pashaei et al. [ 25 ] proposed two different methods: in the first method, a CNN model was used for classification, and in the second method, the features extracted by CNN were the inputs of a KELM method. The KELM method is a learning algorithm that consists of layers of hidden nodes. Sultan et al. [ 26 ] proposed a CNN network including 16 convolution layers. Finally, Anaraki et al. [ 27 ] presented a hybrid method combining the use of CNNs and genetic algorithm (GA) criteria to improve the network architecture.

In this paper, we present a fully automatic CNN-based segmentation and classification method of MRI images for three types of tumor: meningioma, glioma, and pituitary tumors. Unlike previous CNN-based works [ 10 , 23 , 24 , 25 , 26 , 27 ], our neural network includes three processing pathways to deal with the information in three spatial scales. Previous research conducted by our research group and inspired in the inherent multiscale operation of the Human Visual System (HVS) [ 28 ] led us to propose this multiscale processing strategy for tumor classification on the assumption that a multiscale approach would be effective extracting discriminant texture features of different kinds of tumors. The proposed model is very coherent with the HVS processing. In visual stimuli processing, the HVS works in two main modes: pre-attentive and attentive vision. Pre-attentive mode is instantaneous and parallel, covering wide areas of the vision field, while attentive processing acts over limited regions of the visual field, establishing a serial search through focused attention [ 29 ]. In this processing, simple cells in the V1 area of the visual cortex in the brain filter the stimuli coming from the LGN (Lateral Geniculate Nucleus), through frequency and bandwidth filtering. After that filtering, the processed features are concatenated in the inferotemporal area during the cognitive process [ 30 ]. In addition, our method uses a sliding window technique differing from U-net [ 13 ] and other methods, in which the full image is used as the input.

The rest of the paper is organized as follows: Section 2 introduces the proposed CNN model, describes the adopted dataset and how it was used for training and measuring the performance of the neural network. Later, Section 3 presents the experimental results and discusses these results including a performance comparison with other methods. Finally, Section 4 draws the main conclusions about the presented work.

2. Materials and Methods

2.1. proposed convolutional neural network and implementation details.

In this paper, we propose a multi-pathway CNN architecture (see Figure 2 ) for tumor segmentation. The CNN architecture processes an MRI image (slice) pixel by pixel covering the entire image and classifying each pixel using one of four possible output labels: 0—healthy region, 1—meningioma tumor, 2—glioma tumor, and 3—pituitary tumor.

The proposed Convolutional Neural Networks (CNN) architecture. Input: 1 × 65 × 65 sliding windows. Model: Three pathways (large, medium, and small feature scales) with 2 convolutional layers and max-pooling, a convolutional layer with concatenation of the three pathways, and a fully connected stage that leads to a classification in one out of the four possible output labels: 0—healthy region, 1—meningioma tumor, 2—glioma tumor, and 3—pituitary tumor. A dropout mechanism between the concatenation and fully connected stages is included.

In our approach, we use a sliding window, thus each pixel is classified according to a N × N neighborhood or window, which is the input to our CNN architecture (see Figure 2 ). Every window is processed through three convolutional pathways with three scale (large, medium, and small) kernels that extract the features. In our implementation, we chose a window of 65 × 65 pixels and kernels of size 11 × 11, 7 × 7 and 3 × 3 pixels, respectively. The decision on the size of the windows was taken after preliminary configuration tests in which also 33 × 33 px and 75 × 75 px window sizes were tested.

Each pathway is composed by two convolutional stages with ReLU rectification and a 3 × 3 max-pooling kernel with a stride value of 2. The number of feature maps in the large, medium, and small pathways is 128, 96, and 64, respectively. We propose the use of a larger number of maps for larger scales with the assumption that the window features extracted using different filtering scales help to define the three kinds of tumor to be classified.

Scale features from the three pathways are concatenated in a convolutional layer with a 3 × 3 kernel with a ReLU activation function and a 2 × 2 max-pooling kernel with a stride value of 2. The output of this stage enters a fully connected stage where 8192 concatenated scale features compose the classification method towards the four prediction label types. In order to prevent overfitting, the model includes a dropout layer before the fully connected layer. The last layer uses a softmax activation function.

The proposed CNN has been implemented using Pytorch™. The number of trainable parameters in the neural network is near three million (2,856,932). All the tests have been performed in a Linux environment with an Intel Core i7 CPU and an Nvidia GTX1080 T1-11GB GPU. The training process took 5 days, and the average prediction time per image was 57.5 s.

2.2. Dataset and Data Preprocessing

In clinical settings, usually only a certain number of slices of brain CE-MRI with a large slice gap, not a 3D volume, are acquired and available. It is difficult to build a 3D model with such sparse data. Hence, the proposed method is based on 2D slices collected from 233 patients, acquired from Nanfang Hospital, Guangzhou, China, and General Hospital, Tianjing Medical University, China, from 2005 to 2010 [ 5 ].

This dataset contains 3064 slices and includes meningiomas (708 slices), gliomas (1426 slices), and pituitary tumors (930 slices) in the common views (sagittal, coronal, and axial). Figure 1 shows examples of these three types of tumors. This dataset provides also 5-fold cross-validation indices. By using this information, 80% (2452) of the images are employed for training and 20% (612 images) are used for obtaining performance measurements. The process is repeated 5 times.

The images have an in-plane resolution of 512 × 512 pixels with pixel size 0.49 × 0.49 mm 2 . The slice thickness is 6 mm, and the slice gap is 1 mm. The tumor border was manually delineated by three experienced radiologists. Every slice in the dataset has an information structure attached containing the patient’s pid; the tumor type label ( l g t   ): 1 for meningioma, 2 for glioma, and 3 for pituitary tumor; the coordinates vector ( x , y ) of the points belonging to the tumor border, and the tumor mask ( T i j ): a binary image where the tumor positions contain a value of 1 and the healthy ones a value of 0. The pair l g t   ,   T i j will be the ground truth in the training process.

During training, data augmentation using an elastic transform [ 31 ] has been used to prevent overfitting of the neural network. Figure 3 shows an example of this transformation. Data augmentation procedure doubled the number of training images available on each fold iteration up to 4904 images. A thorough process was conducted to extract 65 × 65 px training examples from every image in the training dataset: 150 true positive window examples and 325 true negative window examples per tumor. The pixels on these windows were scaled using pixel standardization (zero mean and unit variance) across the entire training dataset.

Example of elastic transformation used in the data augmentation. Left: original slice. Right: image transformed. In both images, the edges of the tumor have been highlighted in red.

2.3. Neural Network Training and Performance Measurements

The proposed CNN and performance measurements were obtained after a 5-fold cross validation technique with the slice train/test subgroups specified in the dataset, as explained before.

On each fold iteration, the proposed model was trained for a total of 80 epochs, using a Stochastic Gradient Descent (SGD) optimizer with a starting learning rate of 0.005 and a momentum coefficient of 0.9. In addition, we used a learning rate exponential decay every 20 epochs. The dropout parameter was set to 0.5.

Within each fold, performance of the model was tested over 612 images, using a sliding window approach: every testing window of size 65 × 65 px enters the trained CNN so that a predicted tumor type label l p ,   is obtained. Segmentation of the tumor is achieved considering the region containing all the pixels identified as a given tumor type. The previously stored global mean and standard deviation of pixels in the training windows, computed for data preprocessing, are used to normalize the testing windows before they enter the CNN.

Once all pixels i , j from the input slice are labeled, P i j (see Equation (1)), the classification function can be calculated using Equation (2). The predicted label l p , which indicates the tumor type in a slice, is determined from vector { f l ,   l = 1 , 2 , 3 } in the classification function. The classification function calculates the relation between sizes of the predicted label, l , P i j = = l (number of pixels with the predicted label l ), and the complete prediction, { P i j > 0 } (number of pixels with a predicted label belonging to any tumor). The predicted label, l p ,   will belong to the label with the greatest relation in size that is above the minimal relation defined by the confidence threshold, τ c ∈   0 , 1 .

where . is the size or pixel number (number of pixels in the slice that meet the conditions inside).

For performance measurements, we compute the Confusion Matrix, together with the Dice and Sensitivity scores [ 3 ]. In addition, with the aim of evaluating the precision of the predicted tumor type, we defined the predicted tumor type ratio score, pttas. This index calculates the relation in size between the correctly labeled tumor regions P i j = l g t and all the tumor regions determined by our methodology { P i j > 0 } . In the pttas index, the ratio is measured over the total size of the predicted tumor, while in the Sensitivity index the ratio is calculated over the size of the ground truth tumor.

The evaluation metrics are calculated using the following Equations:

where TP is the number of true positives, FP is the number of false positives, FN is the number of false negatives, ∧ is the logical AND operation, .   is the size or pixel number (number of pixels in the slice that meet the conditions inside), P 1 represents the addition of TP and FP (the predicted positives P i j =   l g t ), and T 1   represents the addition of TP and FN T i j = 1 .

3. Results and Discussion

As explained in the previous section, the neural network was tested using the 5-fold cross validation train/test subgroups specified in the dataset. The quantitative results obtained are displayed in the figures and tables below in this section.

Figure 4 shows the performance of our method for three slices. Figure 4 —left—corresponds to a slice with a meningioma tumor, Figure 4 —center—shows a glioma tumor, and Figure 4 —right—a pituitary tumor. The images show the tumor segmentation obtained, { P i j > 0 } . To mark them with different colors, color images were generated where the predicted tumor region is filled with red color, the ground truth tumor region, T i j , is colored green, and the intersection of those two regions appears in yellow.

Examples of results of the proposed method for three slices corresponding to meningioma, glioma, and pituitary tumors, respectively. The images show the tumor segmentation: The region detected is shown in red while the ground truth region is shown in green. As a result, the intersection region is shown in yellow.

The segmentation metrics obtained are shown in Table 1 . The Dice index, which is inversely related to the number of false positives and false negatives extracted (see Equation (3)), is the largest for images with meningioma, then with pituitary tumor, and the lowest with glioma. Therefore, the proportion of false positives and false negatives is the lowest in the prediction of meningiomas and is the largest in the prediction of gliomas. The Sensitivity index, which computes the ratio between true positives and the ground truth (see Equation (4)), led to the same ranking (the largest value for images with meningioma, then with pituitary tumor, and the lowest value with glioma), although the value of this index is closer for images with meningioma and pituitary tumor. The prediction of gliomas had the lowest relation of true positives with respect to the ground truth. On the contrary, the pttas index is the largest for images with glioma, then with pituitary tumor, and the lowest with meningioma. The values of this index, which measures the relation in size between the correctly labeled tumor regions and all the predicted tumor regions in the images, indicates that for gliomas our model misclassified a lower proportion of pixels as belonging to the two other brain tumors than for meningiomas or pituitary tumors. It can be observed that the different features of the three analyzed brain tumors and the terms including in the segmentation metrics led to a different ranking in the segmentation of the three tumors. The values obtained for all the segmentation metrics are remarkable with average values of Dice = 0.828, Sensitivity = 0.940, and pttas = 0.967.

Segmentation metrics of the 3064 slices processed with 5-fold cross validation.

The histograms for these metrics of the segmentation per processed slice are included in Figure 5 . Remarkably, Sensitivity and pttas histograms have their values concentrated on a very narrow interval around the highest zone. This means that almost all the processed images have a high value in those evaluation metrics.

Histograms of metrics obtained after processing the dataset. Left: Dice histogram. Center: Sensitivity histogram. Right: Histogram of the predicted tumor type.

Regarding the pttas index histogram (see Figure 5 —right), we can see a very outstanding behavior. Around value 0, there are just a few samples. In these samples the predicted tumor is wrong as pttas is lower than 1/3 (pttas measures the ratio to the right label). If pttas is larger than 1/3, it means that the tumor type prediction was correct. The rest of the samples are found in values above 0.4, which means that they have all been correctly predicted. The high values of the Sensitivity index (see Figure 5 —center) are due to the fact that almost all the tumor regions have been correctly extracted in the processed images (regions in yellow in Figure 4 ). A greater amplitude in the Dice histogram (see Figure 5 —left) reflects the existence of false positives and false negatives. However, the Dice index results are relatively high, so the number of false positives and false negatives is quite low.

Figure 6 shows some segmentations with misclassified areas; in the example of the upper row, the greater part of the detected tumor is correctly labeled (red area in the right column), resulting in a very correct segmentation (yellow area in the left column). However, there is a region detected in the non-cerebral area wrongly labeled as glioma tumor (green area in the right column). This example shows the added complexity inherent to this dataset due to the fact that it includes non-cerebral areas that can generate false positives. This complexity also manifests itself in the example of the middle row. Similarly, the segmentation is relatively correct (yellow region in the left column), but there is a misclassified area labeled as a pituitary tumor (blue region in the right column) located in the sphenoidal sinuses area, which is where pituitary tumors appear. The physical structure of the sphenoidal sinuses led to confusion to our model. The third example (lower row) shows a confusion between a real glioma (green region in the right column) and a wrongly predicted meningioma region (red area in the left column).

Confusion in segmentation. Color-generated images for proper visualization of regions. Upper row corresponds to a meningioma tumor, middle row to a meningioma tumor, and lower row to a glioma tumor. Right column shows the predicted tumor tags (red: meningioma, green: glioma, blue: pituitary tumor). Left column shows segmentations (red: predicted, green: ground truth and, as a result, the intersection is shown in yellow).

In our method, tumor classification is a direct outcome of the resulting segmentation. The classification function for the tumor in the image counts the tumor type prediction for every pixel (pttas, see Equation (5)) and considers the highest value to be the predicted tumor type given that it is greater than the confidence threshold τ c (see Equation (2)). Table 2 and Table 3 show the values of the confusion matrix and the tumor classification accuracy for Cheng et al. [ 5 ] method and our method, respectively. It can be observed that the confusion matrix for our method has very high values in its diagonal, which results in a high tumor classification accuracy of 0.973.

Confusion matrix of the predicted tumor for Cheng et al. [ 5 ] method. The tumor type precision (tumor classification accuracy) achieved ( ∑ diag / ∑ total ) is 0.912.

Confusion matrix of the predicted tumor for our method. The number of non-classified images is 61 (42/4/15), taking a confidence threshold τ c = 0.75 . The tumor type precision (tumor classification accuracy) achieved ( ∑ diag / ∑ total ) is 0.973.

The classification function, Equation (2), depends on the confidence threshold parameter τ c . Figure 7 shows the relationship between the confidence threshold and the resulting precision. A slow descent can be observed up to a confidence threshold of 0.88 followed by a sharp decline. There is no strong restriction on the selection of the confidence threshold and, in order to obtain a classification with a precision level higher than 0.9, it could be possible to raise our confidence threshold to a very high value (around 0.92).

Graph of the relation between the confidence threshold and the precision of the predicted tumor type.

This behavior could be predicted observing the pttas index histogram (see Figure 5 ). Peak values in the histogram can be found once a pttas value of 0.88 is reached. The number of images with a pttas index value between 0.4 and 0.88 is very low, so when the confidence threshold is raised from 0.4 to 0.88, the precision will only be marginally lower as can be observed in the graph in Figure 7 .

Due to the fact that all the processed images with a pttas index value greater than 0.4 are correctly labeled, if we were to choose a confidence threshold equal or lower than 0.4, the precision would be close to 100% (all samples would be correctly labeled, except those with a value close to zero). This is why the confidence threshold diagram begins with accuracy ≈1 (= 0.994).

Comparison with Other Methods

Segmentation performance metrics of our method (see Table 1 ) are in the range of the wining proposals in the well-known brain tumor image segmentation challenge BRATS [ 3 ]. This benchmark uses glioma tumor images and supplies a dataset with four modalities of MRI (T1, T2, T1-CE, and FLAIR). The 2013 top-ten ranking of the participating methods obtained glioma Dice index values between 0.69 and 0.82 for the segmentation of the complete tumor. The Dice index value obtained by our method for glioma segmentation is 0.779, which is close to the highest achieved value, even with the inconveniences associated with working with a single MRI modality and classifying among three different kinds of tumors.

A more appropriate comparison of our results is with those results obtained by previous works using the same T1-CE MRI image dataset. Table 4 shows the tumor classification accuracy of our approach, together with seven published methods: two feature-driven approaches [ 5 , 32 ] and five deep learning approaches [ 24 , 25 , 26 , 27 ]. Our method outperforms the other proposals with a tumor classification accuracy of 0.973.

Comparison of the proposed approach with other approaches over the same T1-CE MRI image dataset.

The two studies conducted by Cheng et al. [ 5 , 32 ] are examples of classical machine learning approaches using pre-defined feature extraction. In a first paper, Cheng et al. [ 5 ] proposed a classification method based on the technique of splitting the augmented tumor region in ring-shaped sub-regions and extracting local features using three different methods: intensity histogram, gray level co-occurrence matrix (GLCM), and bag-of-words (BoW). Those features were employed to create a dictionary with the K-means technique. Then, a trained classifier was used to classify the ROIs as a tumor type. Three classification methodologies were compared: SVM, sparse representation-based classification (SRC), and k-Nearest Neighbors (kNN). The best results were obtained using BoW and SVM as the classifier. In a second paper, Cheng et al. [ 32 ] proposed a content-based image retrieval for MRI images using the Fisher kernel framework. Cheng et al. methodologies [ 5 , 32 ] start with segmentations performed by experienced radiologists, and thus, they are semi-automatic methods. They need an interactive human intervention during classification, whereas our proposed methodology is fully automatic. Nonetheless, using a confidence threshold of 0.75, our method obtains a tumor classification accuracy of 0.973 versus 0.912 in [ 5 ] and 0.947 in [ 32 ]. Moreover, Figure 7 shows that it is possible to raise the confidence threshold up to 0.9 in order to obtain even a higher classification accuracy. Setting a threshold of 0.75, 61 slices resulted unclassified (2% of the total processed MRI images): 42 meningiomas, 4 gliomas, and 15 pituitary tumors. These figures show to what extent the detection of meningioma tumors is much more demanding. As in Cheng et al. [ 5 ] method (see Table 2 ), in our work the lowest Sensitivity value is for meningioma (see Table 3 ), due to the lower intensity contrast between tumor areas and healthy areas. Sensitivity for glioma is the highest. Pituitary tumors are somewhere in between meningiomas and gliomas.

The five Deep Learning methods in Table 4 also obtained a lower accuracy than our proposal. Abiwinanda et al. [ 24 ] used two convolution layers and 64 fully connected neurons. They did not use the full dataset but selected 700 images from each brain tumor type, with the purpose of balancing the data. A subset of 500 tumor images of each class was used for training and a subset of 200 images for testing. They did not apply data augmentation. Their classification accuracy was 0.841, quite lower than our approach (0.973). Pashaei et al. [ 25 ] proposed two Deep Learning methods. In their first method, a CNN model is used for classification with 4 convolution and normalization layers, 3 max-pooling layers, and a last fully connected layer. In their second method, the features extracted by a CNN are the inputs of a KELM classifier. They used 70% of the dataset for training, no data augmentation and 30% for testing using a 10-fold cross validation. The classification accuracy was 0.810 with the first method, and it was significantly improved with the second method (0.936). Sultan et al. [ 26 ] used a CNN containing 16 convolution layers, including pooling and normalization, and a dropout layer before the last fully connected layer. They used 68% of the images for training and the remaining 32% for validation and test. Significantly, training data was increased by a factor of 5 with geometric augmentation and grayscale distortion. Their classification accuracy is the second highest in Table 4 (0.961). Finally, Anaraki et al. [ 27 ] also proposed a CNN but included a genetic algorithm to select the best configuration of the CNN: number of convolutional, max-pooling and fully connected layers, activation function, dropout parameter, optimization method, and learning rate. Data augmentation was carried out by applying rotation, translation, scaling, and mirroring. They obtained an accuracy of 0.942.

There are some other interesting deep learning approaches that used a different brain tumor MRI dataset in their experiments. Havaei et al. [ 10 ] trained several CNN neural networks with the BRATS-2013 glioma tumor dataset. They included a three-step preprocessing: removal of the highest and lowest intensities, intensity inhomogeneity correction, and normalization. The segmentation results were close to those obtained with our method, with a Dice index between 0.77 and 0.88 for the complete tumor and a Sensitivity index between 0.75 and 0.96. Our method obtains a slightly higher Dice index value of 0.779 and a Sensitivity index of 0.907 for glioma segmentation (see Table 1 ). Mohsen et al. [ 23 ] proposed a Deep Learning approach together with DWT and PCA and applied it to classify a dataset of 66 MRIs into 4 classes: normal, glioblastoma, sarcoma, and metastatic bronchogenic carcinoma tumors. They used Fuzzy C-means clustering to separate brain tissues from skull before the deep learning processing. Although the types of brain tumors in the images are different from the ones in the T1-CE MRI image dataset we used, the classifier had to obtain an output label among four different values, similarly to our approach with the adopted T1-CE MRI image dataset. They obtained a classification accuracy of 0.969, lower than our approach.

4. Conclusions

In this paper, we present a fully automatic brain tumor segmentation and classification method, based on a CNN (Convolutional Neural Network) architecture designed for multiscale processing. We evaluated its performance using a publicly available T1-weighted contrast-enhanced MRI images dataset. Data augmentation through elastic transformation was adopted to increase the training dataset and prevent overfitting. The measures of performance obtained are in the range of the top ten methods from the BRATS 2013 benchmark. We compared our results with other seven brain tumor classification approaches that used the same dataset. Our method obtained the highest tumor classification accuracy with a value of 0.973. Our multiscale CNN approach, which uses three processing pathways, is able to successfully segment and classify the three kinds of brain tumors in the dataset: meningioma, glioma, and pituitary tumor. In spite of the fact that skull and vertebral column parts are not removed and the variability of the three tumor types, which caused false positives in some images, our approach achieved outstanding segmentation performance metrics, with an average Dice index of 0.828, an average Sensitivity of 0.940, and an average pttas value of 0.967. Our method can be used to assist medical doctors in the diagnostics of brain tumors and the proposed segmentation and classification method can be applied to other medical imaging problems. As future work, we plan to develop an FCN architecture for the classification of the same MRI images dataset and compare its performance with the proposed model. Moreover, we plan to study the applicability of the proposed multiscale convolutional neural network for segmentation in other research fields such as satellite imagery.

Author Contributions

Conceptualization, F.J.D.-P.; methodology, F.J.D.-P.; software, M.M.-Z., M.A.-R., and D.G.-O.; experimental results, M.M.-Z., M.A.-R., and D.G.-O.; supervision, F.J.D.-P.; writing—original draft and review, F.J.D.-P., M.M.-Z., M.A.-R., and D.G.-O. All authors have read and agreed to the published version of the manuscript.

This research received no external funding.

Institutional Review Board Statement

Not applicable.

Informed Consent Statement

Data availability statement, conflicts of interest.

The authors declare no conflict of interest.

Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Accessibility Links

• Skip to content
• Skip to search IOPscience
• Skip to Journals list
• Accessibility help
• Accessibility Help

Click here to close this panel.

Purpose-led Publishing is a coalition of three not-for-profit publishers in the field of physical sciences: AIP Publishing, the American Physical Society and IOP Publishing.

Together, as publishers that will always put purpose above profit, we have defined a set of industry standards that underpin high-quality, ethical scholarly communications.

We are proudly declaring that science is our only shareholder.

Brain tumor detection from MRI images using deep learning techniques

P Gokila Brindha 1 , M Kavinraj 2 , P Manivasakam 2 and P Prasanth 2

Published under licence by IOP Publishing Ltd IOP Conference Series: Materials Science and Engineering , Volume 1055 , International Virtual Conference on Robotics, Automation, Intelligent Systems and Energy (IVC RAISE 2020) 15th December 2020, Erode, India Citation P Gokila Brindha et al 2021 IOP Conf. Ser.: Mater. Sci. Eng. 1055 012115 DOI 10.1088/1757-899X/1055/1/012115

Article metrics

15757 Total downloads

Share this article

Author e-mails.

[email protected]

Author affiliations

1 Assistant Professor, Department of Computer Technology - UG, Kongu Engineering College, Perundurai - 638 060

2 Student, Department of Computer Technology - UG, Kongu Engineering College, Perundurai - 638 060

Buy this article in print

Brain tumor is the growth of abnormal cells in brain some of which may leads to cancer. The usual method to detect brain tumor is Magnetic Resonance Imaging(MRI) scans. From the MRI images information about the abnormal tissue growth in the brain is identified. In various research papers, the detection of brain tumor is done by applying Machine Learning and Deep Learning algorithms. When these algorithms are applied on the MRI images the prediction of brain tumor is done very fast and a higher accuracy helps in providing the treatment to the patients. These prediction also helps the radiologist in making quick decisions. In the proposed work, a self defined Artificial Neural Network (ANN) and Convolution Neural Network (CNN) is applied in detecting the presence of brain tumor and their performance is analyzed.

Export citation and abstract BibTeX RIS

Content from this work may be used under the terms of the Creative Commons Attribution 3.0 licence . Any further distribution of this work must maintain attribution to the author(s) and the title of the work, journal citation and DOI.

Brain Tumor Detection Using Deep Learning

• Find this author on Google Scholar
• Find this author on PubMed
• Search for this author on this site
• For correspondence: [email protected] [email protected]
• ORCID record for Ewerton Santos
• Info/History
• Preview PDF

A brain tumor is understood by the scientific community as the growth of abnormal cells in the brain, some of which can lead to cancer. The traditional method to detect brain tumors is nuclear magnetic resonance (MRI). Having the MRI images, information about the uncontrolled growth of tissue in the brain is identified. In several research articles, brain tumor detection is done through the application of Machine Learning and Deep Learning algorithms. When these systems are applied to MRI images, brain tumor prediction is done very quickly and greater accuracy helps to deliver treatment to patients. These predictions also help the radiologist to make quick decisions. In the proposed work, a set of Artificial Neural Networks (ANN) are applied in the detection of the presence of brain tumor, and its performance is analyzed through different metrics.

Competing Interest Statement

The authors have declared no competing interest.

Funding Statement

This study did not receive any funding

Author Declarations

I confirm all relevant ethical guidelines have been followed, and any necessary IRB and/or ethics committee approvals have been obtained.

The details of the IRB/oversight body that provided approval or exemption for the research described are given below:

This study involves only openly available human data, which can be obtained from: https://www.kaggle.com/jakeshbohaju/brain-tumor

I confirm that all necessary patient/participant consent has been obtained and the appropriate institutional forms have been archived, and that any patient/participant/sample identifiers included were not known to anyone (e.g., hospital staff, patients or participants themselves) outside the research group so cannot be used to identify individuals.

I understand that all clinical trials and any other prospective interventional studies must be registered with an ICMJE-approved registry, such as ClinicalTrials.gov. I confirm that any such study reported in the manuscript has been registered and the trial registration ID is provided (note: if posting a prospective study registered retrospectively, please provide a statement in the trial ID field explaining why the study was not registered in advance).

I have followed all appropriate research reporting guidelines and uploaded the relevant EQUATOR Network research reporting checklist(s) and other pertinent material as supplementary files, if applicable.

Data Availability

All data produced in the present study are available upon reasonable request to the authors

View the discussion thread.

Thank you for your interest in spreading the word about medRxiv.

NOTE: Your email address is requested solely to identify you as the sender of this article.

Citation Manager Formats

• EndNote (tagged)
• EndNote 8 (xml)
• RefWorks Tagged
• Ref Manager
• Tweet Widget
• Facebook Like
• Google Plus One

Subject Area

• Health Informatics
• Addiction Medicine (316)
• Allergy and Immunology (618)
• Anesthesia (160)
• Cardiovascular Medicine (2277)
• Dentistry and Oral Medicine (279)
• Dermatology (201)
• Emergency Medicine (370)
• Endocrinology (including Diabetes Mellitus and Metabolic Disease) (800)
• Epidemiology (11580)
• Forensic Medicine (10)
• Gastroenterology (678)
• Genetic and Genomic Medicine (3585)
• Geriatric Medicine (337)
• Health Economics (618)
• Health Informatics (2307)
• Health Policy (914)
• Health Systems and Quality Improvement (864)
• Hematology (335)
• HIV/AIDS (753)
• Infectious Diseases (except HIV/AIDS) (13159)
• Intensive Care and Critical Care Medicine (757)
• Medical Education (359)
• Medical Ethics (100)
• Nephrology (389)
• Neurology (3351)
• Nursing (191)
• Nutrition (507)
• Obstetrics and Gynecology (651)
• Occupational and Environmental Health (646)
• Oncology (1759)
• Ophthalmology (525)
• Orthopedics (209)
• Otolaryngology (284)
• Pain Medicine (223)
• Palliative Medicine (66)
• Pathology (438)
• Pediatrics (1005)
• Pharmacology and Therapeutics (422)
• Primary Care Research (406)
• Psychiatry and Clinical Psychology (3063)
• Public and Global Health (5993)
• Radiology and Imaging (1224)
• Rehabilitation Medicine and Physical Therapy (715)
• Respiratory Medicine (811)
• Rheumatology (367)
• Sexual and Reproductive Health (353)
• Sports Medicine (316)
• Surgery (386)
• Toxicology (50)
• Transplantation (170)
• Urology (142)

Brain Tumor Detection Using Deep Learning Models

Ieee account.

• Change Username/Password
• Update Address

Purchase Details

• Payment Options
• Order History
• View Purchased Documents

Profile Information

• Communications Preferences
• Profession and Education
• Technical Interests
• US & Canada: +1 800 678 4333
• Worldwide: +1 732 981 0060
• Contact & Support
• About IEEE Xplore
• Accessibility
• Terms of Use
• Nondiscrimination Policy
• Privacy & Opting Out of Cookies

A not-for-profit organization, IEEE is the world's largest technical professional organization dedicated to advancing technology for the benefit of humanity. © Copyright 2024 IEEE - All rights reserved. Use of this web site signifies your agreement to the terms and conditions.

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

• View all journals
• My Account Login
• Explore content
• About the journal
• Publish with us
• Sign up for alerts
• Review Article
• Open access
• Published: 29 March 2024

Artificial intelligence in neuro-oncology: advances and challenges in brain tumor diagnosis, prognosis, and precision treatment

• Sirvan Khalighi   ORCID: orcid.org/0000-0002-5563-1446 1 ,
• Kartik Reddy 2 ,
• Abhishek Midya 1 ,
• Krunal Balvantbhai Pandav 1 ,
• Anant Madabhushi   ORCID: orcid.org/0000-0002-5741-0399 1 , 3 &
• Malak Abedalthagafi   ORCID: orcid.org/0000-0003-1786-3366 4 , 5  

npj Precision Oncology volume  8 , Article number:  80 ( 2024 ) Cite this article

1204 Accesses

26 Altmetric

Metrics details

This review delves into the most recent advancements in applying artificial intelligence (AI) within neuro-oncology, specifically emphasizing work on gliomas, a class of brain tumors that represent a significant global health issue. AI has brought transformative innovations to brain tumor management, utilizing imaging, histopathological, and genomic tools for efficient detection, categorization, outcome prediction, and treatment planning. Assessing its influence across all facets of malignant brain tumor management- diagnosis, prognosis, and therapy- AI models outperform human evaluations in terms of accuracy and specificity. Their ability to discern molecular aspects from imaging may reduce reliance on invasive diagnostics and may accelerate the time to molecular diagnoses. The review covers AI techniques, from classical machine learning to deep learning, highlighting current applications and challenges. Promising directions for future research include multimodal data integration, generative AI, large medical language models, precise tumor delineation and characterization, and addressing racial and gender disparities. Adaptive personalized treatment strategies are also emphasized for optimizing clinical outcomes. Ethical, legal, and social implications are discussed, advocating for transparency and fairness in AI integration for neuro-oncology and providing a holistic understanding of its transformative impact on patient care.

Similar content being viewed by others

Towards a general-purpose foundation model for computational pathology

Richard J. Chen, Tong Ding, … Faisal Mahmood

Segment anything in medical images

Jun Ma, Yuting He, … Bo Wang

Starfysh integrates spatial transcriptomic and histologic data to reveal heterogeneous tumor–immune hubs

Siyu He, Yinuo Jin, … Elham Azizi

Introduction

Central nervous system (CNS) tumors, whether primary or secondary, exert a significant impact on global health, accounting for over 250,000 reported cases annually, marking them as a substantial global concern 1 , 2 . In 2022, an estimated 26,670 malignant and 66,806 non-malignant CNS tumors were diagnosed in the US population 3 . Notably, glioblastoma, a fast-growing, aggressive, and malignant type of brain tumor, emerges as a primary contributor to morbidity and mortality among adult brain tumors, exhibiting a disconcerting 6.9% 5-year survival rate and contributing to 10,000 annual deaths in the US 4 , 5 . These numbers highlight the current shortcomings in treating brain tumors.

Despite many clinical trials and decades of research, incurable brain tumors with grim prognoses exist, such as the diffuse midline glioma (DMG) seen in children and glioblastoma in adults 6 . This urgency highlights the need for a personalized treatment approach, which may offer the highest likelihood of cure while minimizing potential toxicity to the patient. However, the development of personalized strategies faces hurdles due to the difficulty of generalizing approaches derived from data originating in a solitary institution or a limited consortium of institutions, and restricted access to advanced technologies and clinical trials, primarily concentrated in specialized centers 7 . This becomes a critical concern, especially when contemplating the ethical ramifications associated with developing approaches based on data lacking representation across diverse demographics. Alarmingly, individuals with glioblastoma from lower socioeconomic backgrounds are less likely to undergo O6-Methylguanine-DNA-methyltransferase (MGMT) testing 8 . The absence of MGMT testing may skew predictions and contribute to late-stage diagnoses with larger and more challenging tumors. Moreover, this demographic is less frequently provided with a combination of diverse treatment modalities, leading to lower survival rates 9 .

For a patient suspected of harboring a brain tumor, the assessment typically initiates with a physical exam and neuroimaging, followed by a biopsy or tumor resection in cases where it is feasible, and subsequent histologic and molecular analyses of extracted tissue conducted through pathology. If deemed necessary, serum or cerebrospinal fluid (CSF) biomarker evaluations may also be performed 10 . Following these assessments, the clinical team must decide on optimal therapy, considering the standard of care, ongoing clinical trials, patient comorbidities, and risks of toxicity. Treatment response is monitored longitudinally through serial MRIs and, occasionally, other blood or CSF biomarkers 11 . Decisions regarding brain tumor treatment often involve multidisciplinary meetings between neuro-oncologists, neurosurgeons, neuroradiologists, molecular pathologists, and neuropathologists, underscoring the complexity of these decisions 12 (Fig. 1 ).

a AI augments the capabilities of neuro-oncologists/radiation-oncologists by enabling integrated diagnosis, offering deeper insights into the disease, facilitating precise prognosis by predicting outcomes, and assisting in patient stratification to tailor treatment plans to individual needs. b AI supports neuroradiologists by leveraging MRI images for automated detection and tumor segmentation, identifying molecular subtypes of tumors, providing quantitative measurements, and delivering diagnostic assistance to distinguish tumors from necrotic regions, all while ensuring automated quality checks. c AI aids neurosurgeons during surgery, contributing to surgical margin assessment and offering real-time diagnosis information and guidance, enhancing surgical precision and patient outcomes. d AI assists neuropathologists in the analysis of fresh/FFPE samples, providing automated measurement of features, aiding in tumor classification and grading, improving tumor detection, and delivering comprehensive analysis of cellular and tissue structures through histo-molecular classification. e Handling mutation data, single-cell information, methylation patterns, RNA sequencing, and more, AI empowers molecular pathologists by supporting biomarker identification, pathway identification, treatment response prediction, variant identification, and serving as a diagnosis assistant, streamlining the complex molecular analysis process (Created with BioRender.com).

However, these steps in disease management are ridden with challenges, and errors may lead to patient morbidity and mortality 13 . The challenges include the need for precise disease diagnosis and staging to guide clinical decisions, the continuous monitoring of post-treatment disease progress, which can be complicated by signals from neighboring neural tissue, and the growing significance of identifying genotype patterns 14 . These genotype patterns have a substantial impact on tumor behavior and clinical outcomes 15 . Ultimately, the challenges of managing brain tumors arise from various factors, including the complexities of the brain, limited accessibility for accurate imaging and biopsy procedures, inherent heterogeneity of tumor biology, variable progression rates, individual variability in treatment susceptibility, and relative lack of reliable biomarkers predictive of prognosis 1 , 16 , 17 . The sensitivity of neural tissue to standard treatment modalities, including surgery, radiation, and chemotherapy, further complicates their care 18 .

Artificial intelligence (AI) shows promise as a transformative tool in neuro-oncology, currently addressing challenges across various clinical management stages. In brain tumor management, AI demonstrates its potential across diagnosis, prognosis, and treatment planning by accelerating and enhancing MRI imaging 19 , detecting abnormalities, optimizing workflows, providing accurate measurements, analyzing extensive medical imaging data, and identifying patterns not easily discernible to human observers 20 . It has significantly advanced the field by providing detailed image analysis for diagnostics, tumor grading, prognosis determination, and treatment response assessment. It also facilitates surgical and nonsurgical treatment planning 21 , accelerates drug discovery 22 , and facilitates recurrence monitoring. AI tools can be incorporated into clinical trials, aiming to improve patient outcomes and may provide the path toward personalized therapy 15 , 23 . In clinical neuroimaging, AI plays a crucial role in tasks such as identifying tumor boundaries and types, refining pre-therapeutic planning, and assessing post-therapeutic responses 24 . The capacity for AI to process extensive datasets offers a transformative approach to precision medicine, potentially addressing commonly encountered pain points at all steps of the patient care experience 25 , 26 , 27 (Fig. 1 ). Additionally, it holds promise in ameliorating global healthcare disparities by providing democratized access to diagnostic, prognostic, and therapeutic strategies 28 , 29 .

Recently, there has been growing exploration of integrating AI tools into radiological and pathological workflows, suggesting potential advancements in neuro-oncology 30 , 31 . In brain tumor analysis, AI serves as a comprehensive framework that encapsulates machine learning (ML) and deep learning (DL) techniques, computer vision (CV), and their integration into Computational Biology. ML algorithms within AI contribute to pattern recognition in imaging and genomic data, while DL, a subset of ML, excels in intricate feature extraction. Computer vision, whether through classical image processing techniques or advanced DL methods, interprets visual data for precise medical image analysis. Computational biology leverages AI, ML, and DL to analyze extensive biological datasets, aiding in understanding the genetic and molecular aspects of brain tumors (Supplementary Table 1 ). The synergy among these techniques enhances the depth and accuracy of brain tumor characterization, influencing diagnosis, prognosis, and treatment planning.

In conducting this review, a comprehensive literature search was conducted across several electronic databases. The search was focused on recently published articles, with an emphasis on studies related to AI applications in brain tumor diagnosis, prognosis, and precision treatment. Our search strategy prioritized peer-reviewed articles, systematic reviews, meta-analyses, and landmark studies in the field. This narrative review provides a comprehensive understanding of AI’s pivotal role in managing primary malignant brain tumors, focusing on gliomas. It explores AI applications in brain tumor diagnosis, prognosis, treatment planning, and predictive analytics. Addressing the multifaceted nature of AI in neuro-oncology, we discuss biomarkers, ethical implications, innovative methods, and challenges, including considerations for racial and sex-specific differences within AI applications and efforts to address disparities in current work limitations. What sets our review apart is its explicit focus on integrating AI in radiology, pathology, and genomics for comprehensive brain tumor analysis. Unlike previous papers, our review emphasizes the convergence of AI applications across radiology, pathology, and genomics, providing a holistic approach to brain tumor diagnostics, prognostics, and treatment planning. While many prior reviews have discussed AI in neuro-oncology broadly, they often lack a specific emphasis on the synergistic integration of AI across these critical domains. Concentrating on diagnostic, prognostic, and treatment planning within the imaging domain, our paper not only explores the latest advancements in AI tailored to pathology, radiology, and genomics but also addresses the gaps left by previous reviews in fully comprehending the interconnected roles of these disciplines in brain tumor management. This focused approach contributes a unique perspective, detailing AI’s transformative role in refining imaging-based diagnoses, prognoses, and treatment planning, which were not thoroughly covered by the broader, less-specialized reviews in the field.

Data types and datasets for brain tumor analysis

Brain tumor analysis relies on a range of data types that are effectively utilized by AI algorithms to unveil crucial characteristics. Key data categories encompass imaging data, genomic data, and clinical data. Medical imaging techniques, such as MRI and CT, offer the opportunity to extract intricate visual features about tumor size, location, morphology, and texture. The current standard for brain tumor imaging involves multi-parametric MRI, including sequences like pre- and post-contrast T1-weighted, T2-weighted, fluid-attenuated inversion recovery (FLAIR), diffusion-weighted (DWI), and susceptibility-weighted imaging (SWI) as commonly obtained sequences. High-volume neuro-oncology centers often incorporate additional techniques like MR spectroscopy and perfusion imaging 11 , 32 . Beyond standard imaging, radiomics extracts quantitative features, while histopathological data, derived from biopsies or surgical resections, encompasses tumor cell morphology and tissue architecture. Genomic data, derived from DNA and RNA sequencing, transcriptomic analysis, and methylation analysis, aids in classifying subtypes and predicting tumor aggressiveness 33 . Moreover, prominent molecular biomarkers play a pivotal role in discriminating between brain tumor subtypes 1 , 9 , 34 . These include mutations in IDH for astrocytomas and oligodendrogliomas, TERT promoter mutations for glioblastomas, EGFR amplification for glioblastomas, gain of chromosome 7 and loss of chromosome 10 for glioblastomas, ATRX mutations for astrocytomas, MGMT promoter methylation for glioblastomas, co-deletion of 1p and 19q chromosomes for oligodendrogliomas, and distinct molecular subtypes for medulloblastoma (MBs) 9 (See Tables 2 and 3 ). Lastly, Clinical data, inclusive of patient history, medical records, and treatment responses, contributes to a comprehensive diagnostic profile, with outcomes data serving as a crucial reference for survival prediction models (see Table 1 ).

In addition to more standardly employed data types, innovative approaches like liquid biopsies have emerged for the early detection of brain tumors 35 . Circulating tumor DNA (ctDNA) analysis, a non-invasive method, monitors tumor mutations and genetic changes through fragments of tumor DNA in the bloodstream 36 , 37 . The integration of these diverse data types and advanced technologies enables a new era of accurate, minimally invasive, and effective approaches for diagnosing and treating brain tumors, overcoming the limitations of conventional diagnostic methods. The integration of multiple data sources through multimodal data fusion enhances analyses accuracy by offering a more comprehensive view of the tumor’s characteristics and behavior 38 (Fig. 2 ). A concise overview of each data type, including its description and purpose, is provided in Table 1 .

Shown is the structural framework of a multimodal integration method designed to improve brain tumor management. The process involves the assimilation of data from five different sources, each contributing unique information. From MRI scans, radiomic data is generated. This data includes segmented MRI images achieved through AI-driven segmentation techniques, providing information about the tumor’s spatial characteristics. Blood samples yield ctDNA, allowing for the extraction of epigenomic, fragmentomic, and genomic alterations that inform the molecular landscape of the tumor. CFS samples provide cell-free DNA (cfDNA), offering genomic alteration information and contributing to a comprehensive understanding of the tumor’s genetic profile. Formalin-fixed paraffin-embedded (FFPE) tissue samples provide transcriptomic and molecular pathology data, offering information about gene expression and cellular structure. Clinical information such as age, race, gender, and electronic medical records (EMR) data supplement the molecular and imaging data, enriching the patient’s profile. For each of these modalities, feature extraction is performed, generating a set of informative characteristics. Subsequently, predictive models are applied to each dataset to estimate key outcomes related to diagnosis, prognosis, and treatment response. In the late multimodal integration, the predictions from these distinct models are fused to improve performance and precision. By synthesizing information from diverse sources and modalities, the integrated approach enhances the reliability and accuracy of neuro-oncological diagnosis, prognosis, and treatment response prediction (Created with BioRender.com).

In addition to the institutional datasets, numerous public datasets play a crucial role in evaluating AI-based algorithms for brain tumor diagnosis, prognosis, and treatment planning. Tailored to diverse research needs, these datasets cover various aspects of the disease. The Cancer Imaging Archive (TCIA) is notable among general brain tumor datasets, offering a comprehensive repository of medical imaging data, including MRI, CT, and PET scans for various tumor types 39 . The MICCAI BraTS Challenge provides standardized brain tumor segmentation datasets annually, ideal for assessing algorithms focused on precise tumor delineation 40 .

For specific tumor types, resources such as the NCI TARGET dataset include dedicated sections for glioblastoma (TCGA-GBM) and lower-grade gliomas (TCGA-LGG) 41 . Additional platforms such as the open data alliance and the NCI data commons offer open-access datasets across scientific domains, including medical and brain tumor datasets 42 , 43 . Selecting the most appropriate dataset depends on factors like tumor type, imaging modality, data type (MRI, CT, PET, etc.), availability of ground truth annotations, and data size, allowing researchers to align their choice with specific research interests for AI-driven investigations into brain tumor diagnosis and treatment planning.

Advancements in AI-enhanced preprocessing for precision brain tumor analysis

In brain tumor analysis, AI has addressed the challenges of navigating brain anatomy and tumor variability and significantly enhanced crucial preprocessing steps for accurate diagnosis, prognosis, and treatment planning. Addressing issues of spatial consistency, AI-powered algorithms, such as those integrated into the BrainNet viewer, correct artifacts and distortions in MRI images 44 . This correction facilitates more precise tumor localization and segmentation, which is crucial for effective brain tumor analysis.

Moreover, AI streamlines the intricate process of tumor localization with remarkable accuracy, as demonstrated by algorithms evaluated on the BraTS dataset 40 . Notably, some AI-empowered methods have achieved high accuracy in localizing tumors, thereby enhancing efficiency for radiologists and reducing the potential for human error 45 , 46 . Image segmentation via CNNs, which are adept at uncovering complex patterns from data, has emerged as a powerful tool 47 , 48 , 49 , 50 , 51 . AI-driven algorithms, including the nnU-Net algorithm 52 , demonstrate exceptional proficiency in automating the crucial task of segmenting normal tissues in medical images. This segmentation is vital for tumor analysis, assisting radiologists in identifying areas to avoid during radiation therapy or surgery 52 . In recent developments, the federated learning framework has demonstrated comparable or superior results in the automated segmentation of rare pediatric tumors from MRI images. This approach leverages data from diverse institutions while ensuring the utmost confidentiality of patient information 24 .

The integration of multimodal data enhances detection efficacy by tapping into diverse information sources 53 . The landscape of DL introduces innovative architectures, with some notable exemplars, including the 3D U-Net 46 , DeepMedic 54 , and V-Net 55 . The 3D U-Net, designed to excel in segmenting 3D images of glioblastoma, and recognized for its straightforward training and consistent effectiveness, achieves remarkable results in brain tumor segmentation. DeepMedic 54 , known for its robustness in managing noise and artifacts, stands as a strong competitor to the 3D U-Net, trained on glioma images. The V-Net, a nascent innovation designed to accurately segment volumetric medical images, establishes its prowess in segmenting both 2D and 3D MRI images 55 . Overall, AI’s role in these preprocessing steps empowers radiologists to conduct brain tumor analysis with heightened precision and efficiency.

AI in brain tumor diagnosis

Brain tumor diagnosis involves the identification and characterization of abnormal growths or masses within the brain, utilizing various medical imaging, pathological, and clinical methods to determine the nature, location, and characteristics of the tumor 55 . Brain tumors vary based on their origin, location, histology, malignancy, and patient age, and categorizing them is crucial for diagnostics, prognosis, and treatment planning 55 . These tumors consist of diverse subtypes, each characterized by distinct cellular origins and histological features. While pediatric-type low and high-grade gliomas and MB are most common in children, glioblastomas, diffuse gliomas, and meningiomas predominate in adults 56 . Given their significance, the predominant focus of research has been on glioblastoma and other diffuse glioma in adults and MB, and pediatric type low- and high-grade glioma in children 57 , 58 . Accurate identification of the tumor subtype empowers clinicians to customize diagnostic methods, predict disease behavior, and inform targeted therapies 59 . Unique genetic and histological signatures associated with different tumor types provide information about aggressiveness and treatment responses, guiding the selection of imaging modalities, biomarker assessments, and treatment plans 60 .

Brain tumors are often identified on CT performed in the emergency room setting, prior to being further characterized via MRI, and are definitively diagnosed via histopathologic examination 56 . Standard of care in neuropathology now includes molecular and genetic testing for many tumor types, guided by initial histologic findings 57 . Current diagnosis methods face challenges 58 , 59 , 60 such as early detection due to tumor concealment 61 , imaging limitations 20 , and issues of visualizing small or deep-seated tumors 62 , 63 , difficulties in distinguishing tumor types 64 , invasive procedures with associated risks 65 , and the heterogeneity of brain tumors 66 . These approaches are hindered by time-consuming processes, limited accessibility, and interpretation variability among experts, highlighting the need for advanced AI-based methods 16 , 67 .

AI can add value at all steps of tumor diagnosis, with the majority of current studies attempting to create predictive models trained using imaging data, pathology data, or both data modes combined 68 . The integration of AI models in brain tumor diagnosis shows potential, particularly in distinguishing between glioma and solitary brain metastases using quantitative image analysis methods 69 , 70 . Neuroimaging provides a unique glimpse into the unaltered tumor in its entirety, while pathologic analyses provide a more in-depth look at the cellular and molecular features of the tumor. ML, as a key element of AI, is contributing to advancements in brain tumor diagnostics by enhancing accuracy, expediting image analysis, enabling early detection, and improving differentiation between tumor types. Recent progress in CV, ML, and DL holds the potential for addressing challenges and improving patient care in brain tumor diagnosis 45 .

AI-empowered radiology, and histology-based diagnostic methods

Radiology- and histology-based brain tumor diagnosis involves extracting quantitative features from medical images such as MRI scans or H&E WHI to capture tumor morphology, texture, and spatial relationships. Traditional feature extraction approaches such as texture analysis (Gray-level co-occurrence matrix (GLCM), Gray-level run-length matrix (GLRLM), and Haralick 71 , 72 ), Shape analysis 73 , Intensity analysis 74 , Wavelet-based analysis 75 , are complemented by newer approaches. Then, ML and DL construct predictive models, enabling a personalized, data-driven approach to diagnosis 31 , 76 , 77 . Common techniques comprise CNNs, RNNs, vision transformers, generative adversarial networks (GANs) 78 , support vector machines, and random forests 77 (Supplementary Table 1 ). Models generated from multiple sequences, such as mpMRI have been shown to be more accurate when compared to single sequence models 79 for tumor detection, assessing grades, and guiding treatment planning 80 .

The histology-based diagnosis methods, aligned by the 2021 WHO (World Health Organization) CNS classification book, are central to brain tumor pathology. The histology-based analysis encompasses various molecular techniques leveraging histological data to enhance brain tumor diagnosis. Methylome profiling, a recently influential technique using AI/ML-based classifiers, has become an influential technique for categorizing and diagnosing brain tumors 81 . While the 2021 WHO classification supports the use of methylome classifiers for various brain tumors either as essential or desirable criteria, there’s ongoing debate over the best method and the limited accessibility of diagnostic tests 34 . DeepGlioma 82 , an AI-based diagnostic screening system, offers rapid results (<90 seconds) by streamlining the molecular diagnosis of GMGs using stimulated Raman histology (SRH) images. This innovative system has been developed and validated on a multicenter cohort, highlighting its potential for rapid and accurate brain tumor diagnosis 82 .

Furthermore, innovative approaches utilizing radiomics on MRI perfusion scans demonstrated the ability to predict IDH mutations, providing valuable information for diagnosis and treatment planning 83 . Terahertz spectroscopy has been explored as a non-invasive technique for predicting IDH mutations in glioma tissue samples, presenting a promising alternative to existing methods 84 . Moreover, advanced analysis techniques applied to 18F-FET PET/CT scans have enabled the prediction of both glioma grade and IDH mutation status in untreated patients 85 . Notably, a deep learning imaging signature (DLIS) has been developed, offering accurate prediction of 1p/19q co-deletion in diffuse lower-grade gliomas through pre-operative MRI scans, presenting a non-invasive alternative with significant diagnostic potential 86 .

The Integration of immunohistochemistry, methylation profiling, chromosomal microarray, scRNA-seq 87 , and NGS, with histology-based analysis could further enhance brain tumor diagnosis 81 , 88 . While conventional approaches, utilizing imaging, tissue biopsies, and genetic testing, confidently identify many brain tumors by combining histology with specific genetic changes, exceptions exist, such as high-grade astrocytoma with piloid features, introduced in the 2021 WHO classification 88 , 89 . This particular condition demands methylome profiling for diagnosis 89 , but its rarity suggests that methylome classifiers are best suited for specific cases with atypical clinical and pathological presentations. A recent deep learning method named “Sturgeon,” can rapidly and accurately classify CNS tumor types during surgery using sparse methylation array data obtained from nanopore sequencing produced during surgery. It classifies CNS tumors within 40 minutes after starting sequencing, with an accuracy of 72% in real-time surgical settings 90 . This method allows surgeons to make more informed decisions about the extent of resection, potentially reducing the risk of complications and improving patient outcomes.

AI in brain tumor prognosis

Prognosis in neuro-oncology involves estimating disease progression for an individual, considering treatment planning, disease stage, and site 91 . Key metrics are overall survival (OS), and progression-free survival (PFS), crucial for assessing prognosis and guiding treatment 92 . However, conventional methods relying on disease stage and clinical variables face limitations, including interpretational complexities, biases, and the need for extensive datasets. Achieving precision for personalized care in predicting recurrence and survival remains challenging with conventional methods.

In brain tumor care, AI plays a pivotal role in advancing prognostic capabilities. ML and DL techniques are increasingly being harnessed to predict OS, and PFS, leveraging features extracted from pre-treatment imaging data. Noteworthy studies, including radiomic signatures from T1 and FLAIR MRI scans of glioblastoma patients 69 , and T1, T2, and FLAIR scans from treatment-naïve patients, show significant promise in predicting PFS and OS 93 , 94 . The AI models outperform routine clinical variables and demonstrate excellence when combined with clinical attributes in glioblastoma patients 95 , 96 . Remarkably, models based on T2-weighted MRI 97 and radiomic features from peritumoral edema reveal associations with survival outcomes, site of recurrence, and molecular subtype 98 , especially in glioma 97 , and glioblastoma patients 99 . DL-based models are created to identify tumors and forecast the site of recurrence, sometimes before radiologists can detect it 100 . These models, using various imaging methods, highlight AI’s exceptional predictive capabilities 97 , 99 (Table 2 ).

Furthermore, Intra-tumoral heterogeneity and cell-state plasticity have been identified as key drivers for the therapeutic resistance of glioblastoma 30 . Spatial transcriptional profiles and prognosis from histology images were predicted using this DL framework, shedding light on the potential of AI in unraveling complex aspects of tumor behavior 30 . Additionally, the identification of IDH mutations has been leveraged to guide prognosis, while the definition of glioblastoma has been refined through the analysis of TERT promoter, EGFR amplification, gain of chromosome 7, and loss of chromosome 10. Additionally, H3F3A has emerged as a key marker for aggressive pediatric tumors 101 .

AI in brain tumor therapeutic management

In addition to adding value to both diagnostic and prognostic capabilities, AI has been used for improving brain tumor treatment planning and treatment response assessment 102 . It transforms therapeutic approaches and enhances precision by aiding in the identification and characterization of brain tumors. It guides clinicians in determining the most suitable treatment strategies for individual patients. This multifaceted process involves diverse techniques, including imaging, clinical assessments, biopsies, and molecular analyses, for precise determination of tumor presence, type, location, and extent.

AI-based methods excel in predicting therapy responses, enabling improved treatment planning across various cancers 103 . Novel approaches, such as predicting responses to gamma knife radiosurgery for metastatic brain tumors using radiomic features 104 from contrast-enhanced T1 and FLASH scan and utilizing predictive models based on pre-treatment ADC maps for forecasting responses to radiation therapy, showcase AI’s efficacy 104 . Integrated models, combining radiomics with clinical attributes, effectively assess radiotherapy responses for patients with brain metastasis from primary breast and lung cancer across multicenter patient cohorts 105 . Spatial heterogeneity analysis of peritumoral edema (ED) in glioblastoma aids in identifying high-risk habitats within ED, leading to enhanced treatment planning 106 . While identifying the crucial marker MGMT for temozolomide (TMZ) resistance in glioblastoma patients presents challenges, AI-based radiomic methods emerge as predictors of both MGMT status and TMZ response, providing valuable insights for informed treatment decisions 101 , 107 (Table 3 ).

Integrative multimodal and multiscale analysis

In multimodal and multiscale approaches, the hope is for a more comprehensive understanding of brain tumors through the integration of genomics, pathomics, and radiomics data. Genomics, especially through techniques like NGS, takes a prominent role in unraveling the genetic landscape of brain tumors, providing information into their genomic alterations 88 . Molecular subtyping and biomarkers identified play a critical role in personalized precision medicine, impacting early detection, prognosis, and treatment response prediction. This integrative approach, when combined with clinical data, advances our comprehension and lays the groundwork for tailored treatments targeting specific genetic alterations.

Complementing this genomic foundation, multimodal imaging techniques such as MRI, CT, and PET contribute a rich layer to the integrative tapestry, albeit with challenges of cost and time 108 . When fused with clinical expertize and other diagnostic data, multimodal imaging significantly enhances diagnostic accuracy. The integration of genomics and radiomics, facilitated by AI, emerges as a revolutionary force in understanding and treating brain tumors.

Furthermore, AI’s advancements extend beyond unimodal predictions, ushering in an era of multimodal prognostic and treatment approaches. These multiscale, multimodal approaches extract features from diverse data sources, including radiomic images and multimodal imaging, resulting in a more comprehensive and accurate understanding of the disease trajectory 100 (Fig. 2 ).

Challenges and limitations of AI in brain tumor diagnosis, prognosis, and treatment

Despite the successful integration of AI models in different steps of brain tumor management, challenges persist. These challenges include restricted access to high-quality data, concerns regarding the interpretability and explainability of DL models, and the need for generalizability across diverse populations and tumor types 109 . The reproducibility of radiomic-based features across different institutes faces challenges due to variations in image acquisition parameters, including machines, models, and contrast amounts 110 . Particularly, achieving reproducibility is more complex in MR radiomics compared to CT radiomics 111 . To standardize radiomics, the introduction of the radiomic quality score (RQS) has been pivotal 112 . However, despite the importance of validation of the AI method in neuro-oncology using external dataset, only 29.4% of original studies included external validation 113 .

Additionally, in brain tumors management, it becomes evident that racial disparities introduce intricate dynamics shaped by race, socioeconomic variables, and geographical influences 114 . This complexity extends to various aspects, including recommendations for brain tumor surgery 115 , emphasizing the importance of addressing such disparities in AI-based methods throughout the spectrum of brain tumor management to advance cancer care. Moreover, disparities in brain tumor rates and outcomes, particularly in glioblastoma, manifest differently between males and females 116 . This underscores the necessity for AI-based approaches to factor in sex-related influences across incidence, survival, tumor biology, genetics, treatment response, and prognosis. The key advantages of these models lie in offering enhanced predictions for personalized treatment and the potential for early detection by accounting for gender-specific characteristics.

Ethical, legal, and social implications of AI in brain tumor management

The integration of AI in brain tumor diagnosis, prognosis, and treatment raises critical ethical, legal, and social considerations 117 . Key ethical concerns include ensuring patient privacy through robust data privacy measures, obtaining informed consent, addressing algorithmic fairness, and promoting transparency in AI algorithms and accountability to build and maintain patient trust 28 . Innovations like federated learning aim to tackle the privacy challenge in AI by enabling collaborative model training among multiple parties without the need to share raw data 24 . Ethical imperatives extend to addressing biases and ensuring equitable access. Legal considerations, encompassing liability for AI-generated errors, medical malpractice standards, and regulatory compliance, underscore the need for robust legal frameworks. Collaborative efforts involving policymakers, regulatory bodies, and legal experts are crucial to clarify responsibilities, protect patient safety, and foster responsible AI development. Social implications, such as impacts on patient-doctor relationships, patient empowerment, and healthcare disparities, require careful consideration. AI has the potential to empower patients by providing personalized information and enabling shared decision-making 117 , 118 . However, the equitable access and affordability of AI-driven healthcare need to be addressed to avoid exacerbating existing disparities 117 , 118 .

This review highlights the transformative impact of AI in brain tumor management, signifying a paradigm shift in healthcare that addresses longstanding challenges. AI’s proficiency in ML and DL techniques, specifically in image segmentation, spatial consistency, and prediction, enhances precision in identifying and characterizing brain tumors. This precision contributes to improved diagnostics, prognosis, and personalized treatment planning. The seamless integration of diverse data types, from medical imaging to genomics, along with clinical history, enables a holistic understanding of tumor characteristics, shaping prognosis and personalized treatment plans. AI’s potential to empower clinicians with real-time monitoring, enhanced treatment planning, and optimization is emphasized, promising improved patient outcomes.

The prognostic capabilities of AI-based models surpass routine clinical variables, providing superior predictive accuracy and refining survival predictions 30 . The integration of AI in predicting treatment response, survival time, and site of recurrence is a significant advancement, enabling precise, personalized therapies tailored to individual tumor characteristics and patient-specific data. AI-driven diagnosis not only facilitates real-time monitoring but also improves treatment planning and optimization. This section highlights AI’s potential to deliver more precise, personalized, and effective interventions, contributing to enhanced patient outcomes. Clinical decision support systems, empowered by AI, not only provide evidence-based treatment recommendations but also contribute to ongoing research by generating novel insights and biomarkers.

Ethical considerations in the integration of AI in brain tumor diagnosis, prognosis, and treatment are acknowledged, covering data privacy, algorithmic fairness, legal liability, and social implications. Robust legal frameworks and collaborative efforts such as federated learning are deemed necessary to address these challenges for responsible AI development and societal acceptance in healthcare. Despite challenges such as data collection costs and interpretational complexities, AI integration holds substantial promise, offering prospects for precise and personalized patient care in the future. However, workforce changes and training may be necessary to effectively integrate AI technologies into healthcare settings. The ethical and societal acceptance of AI in healthcare depends on transparent communication, addressing privacy concerns, and promoting fair and inclusive practices.

Overall, AI extends its influence into treatment planning, revolutionizing therapeutic strategies and significantly contributing to improved patient outcomes. The integration of AI into the treatment landscape holds promise for personalized and effective interventions in neuro-oncology.

A vision for the future: a spectrum of approaches

The integration of AI in brain tumor diagnosis, treatment, and prognosis has seen significant progress, yet there are still gaps and promising future directions to explore. Multimodal data integration 108 , real-time monitoring, diagnosis, and adaptive treatment strategies hold the potential to enhance diagnostic accuracy and treatment outcomes 90 . AI can play a critical role in long-term prognostication and survivorship care planning, aiding in treatment decision-making. Bridging the gap between clinical practice and research through data-sharing networks can accelerate AI model development and validation. Transparency and interpretability of AI models are essential for gaining trust and acceptance in clinical settings 119 , 120 . Ethical considerations and human-centered design principles must be prioritized to ensure responsible and patient-centric AI integration. By addressing these aspects, AI has the potential to revolutionize brain tumor care and improve patient outcomes 120 .

Envisioning the future of brain tumor analysis, DL stands at the forefront, but various techniques beckon exploration based on specific applications and data availability. Improved imaging methods, such as functional magnetic resonance imaging (fMRI) 121 and diffusion tensor imaging (DTI) 122 , provide more precise tumor characterization, despite challenges like sensitivity to noise, head motion, magnetic field distortions, and computational expenses 123 , 124 . Computational advancements, notably CNNs and Transformer-based models, enhance accuracy in the detection and classification of brain tumors 125 , 126 , 127 . Multimodal data fusion 108 , encompassing MRI, CT, and PET scans 38 , along with transfer learning using pre-trained models from vast image datasets, addresses the challenge of limited labeled medical data 77 (Supplementary Table 1 ).

Graph-based methods leverage intricate brain region relationships 128 , with graph neural networks (GNNs) and graph-based convolutional networks (GCNs) illuminating the path of modeling brain connectivity and uncovering tumor-associated anomalies 129 . Radiomics and feature engineering extract an array of quantitative features from medical images, with ML algorithms illuminating patterns and correlations. Explainable AI (XAI), a pivotal facet, ensures algorithm transparency and interpretability, a cornerstone in medical applications 130 , 131 . Recent endeavors have gravitated toward devising AI models with explicable outputs, fostering clinicians’ comprehension of decision-making processes, and thus nurturing trust in automated brain tumor detection and classification. Data augmentation and synthesis techniques, encompassing image rotation, scaling, flipping, and the ingenuity of GANs, fortify the training dataset’s robustness 78 , 127 . Additionally, AutoML and hyperparameter optimization tools streamline the optimization of architecture and hyperparameters in brain tumor detection algorithms, culminating in more efficient and precise models.

Collaborative platforms and datasets, such as federated learning 24 , 132 burgeoning repositories of meticulously annotated brain tumor data, expedite algorithmic training, evaluation, and innovation while catalyzing benchmarking efforts. The realm of real-time detection, propelled by advances in hardware such as graphics processing units (GPUs) and field-programmable gate arrays (FPGAs), unlocks the potential for real-time processing of medical images 90 , 133 . Such real-time algorithms hold the promise of streamlining clinical workflows and elevating patient care. The recent utilization of Large Language Models (LLMs) in neurological research demonstrates a remarkable capacity to analyze diverse data sources, offering significant contributions to early diagnosis, patient support, and clinical assistance. Noteworthy challenges, including concerns related to data privacy and biases, highlight the imperative for collaborative endeavors to ensure the responsible development of LLMs in neurology 134 (Supplementary Table 1 ).

Conclusions

We explored the transformative applications of AI, including CV, ML, and DL, in managing brain tumors. AI shows significant promise in diagnosis, prognosis, and treatment planning by effectively detecting and classifying brain tumors from medical images. Through radiomic, pathomic, and genomic analyses, AI contributes to precise tumor characterization. In treatment, AI plays a crucial role in planning, optimization, and response prediction, supporting personalized recommendations and real-time monitoring. The integration of AI-driven approaches aligns with precision medicine and patient-centered care. However, the adoption of AI in brain tumor management requires careful consideration of ethical, legal, and social implications, addressing concerns related to data privacy and healthcare disparities.

Future directions include bridging research gaps, exploring LLM models, multimodal data integration, and advancing real-time monitoring. AI models trained on diverse datasets hold promise for predicting treatment responses and improving patient outcomes. Ongoing development and fine-tuning are essential for uncovering the full potential and challenges in the clinical management of brain tumors, positioning AI as a valuable tool in research and practice.

Brain tumors - classifications, symptoms, diagnosis and treatments. https://www.aans.org/en/Patients/Neurosurgical-Conditions-and-Treatments/Brain-Tumors .

Fan, Y. et al. Burden and trends of brain and central nervous system cancer from 1990 to 2019 at the global, regional, and country levels. Arch. Public Health 80 , 209 (2022).

Ostrom, Q. T. et al. CBTRUs statistical report: primary brain and other central nervous system tumors diagnosed in the United States in 2015-2019. Neuro. Oncol. 24 , V1–V95 (2022).

Article   CAS   PubMed   PubMed Central   Google Scholar  

About Glioblastoma. https://braintumor.org/events/glioblastoma-awareness-day/about-glioblastoma/ . National Brain Tumor Society.

Fekete, B. et al. What predicts survival in glioblastoma? A population-based study of changes in clinical management and outcome. Front. Surg. 10 , 1249366 (2023).

Al Sharie, S., Abu Laban, D. & Al-Hussaini, M. Decoding diffuse midline gliomas: a comprehensive review of pathogenesis, diagnosis and treatment. Cancers (Basel). 15 , 4869 (2023).

Article   PubMed   PubMed Central   Google Scholar  

Acuña-Villaorduña, A., Baranda, J. C., Boehmer, J., Fashoyin-Aje, L. & Gore, S. D. Equitable access to clinical trials: how do we achieve it? Am. Soc. Clin. Oncol. Educ. B . https://doi.org/10.1200/edbk_389838 (2023).

Lamba, N. et al. Socioeconomic disparities associated with MGMT promoter methylation testing for patients with glioblastoma. JAMA Oncol. 6 , 1972–1974 (2020).

Porter, A. B., Wen, P. Y. & Polley, M.-Y. C. Molecular profiling in neuro-oncology: where we are, where we’re heading, and how we ensure everyone can come along. Am. Soc. Clin. Oncol. Educ. B . https://doi.org/10.1200/edbk_389322 (2023).

Xiao, F. et al. Cerebrospinal fluid biomarkers for brain tumor detection: clinical roles and current progress. Am. J. Transl. Res. 12 , 1379–1396 (2020).

Villanueva-Meyer, J. E., Mabray, M. C. & Cha, S. Current clinical brain tumor imaging. Neurosurgery 81 , 397–415 (2017).

Weinberg, B. et al. Nimg-23. Brain tumor reporting and data system (bt-rads) and quantitative tools to guide its implementation. Neuro. Oncol . 21 , vi166 (2019).

Atanasov, A. G. et al. First, do no harm (gone wrong): total-scale analysis of medical errors scientific literature. Front. Public Heal. 8 , 558913 (2020).

Article   Google Scholar  

Aldape, K. et al. Challenges to curing primary brain tumours. Nat. Rev. Clin. Oncol. 16 , 509 (2019).

Bi, W. L. et al. Artificial intelligence in cancer imaging: clinical challenges and applications. CA. Cancer J. Clin . 69 , 127–157 (2019).

Dong, X. et al. 911 Anti-VEGF therapy improves EGFR-vIII-CAR-T cell delivery and efficacy in syngeneic glioblastoma models in mice. J. Immunother. Cancer 11 , e005583 (2022).

Becker, A. P., Sells, B. E., Jaharul Haque, S. & Chakravarti, A. Tumor heterogeneity in glioblastomas: from light microscopy to molecular pathology. Cancers (Basel). 13 , 1–25 (2021).

Stone, J. B. & DeAngelis, L. M. Cancer-treatment-induced neurotoxicity-focus on newer treatments. Nat. Rev. Clin. Oncol . 13 , 92–105 (2016).

Monsour, R., Dutta, M., Mohamed, A. Z., Borkowski, A. & Viswanadhan, N.A. Neuroimaging in the era of artificial intelligence: current applications. Fed. Pract. 39 , S14–S20 (2022).

PubMed   PubMed Central   Google Scholar  

Philip, A. K., Samuel, B. A., Bhatia, S., Khalifa, S. A. M. & El-Seedi, H. R. Artificial intelligence and precision medicine: a new frontier for the treatment of brain tumors. Life . 13 , 24 (2023).

Dundar, T. T. et al. Machine learning-based surgical planning for neurosurgery: artificial intelligent approaches to the cranium. Front. Surg. 9 , 863633 (2022).

Mock, M., Edavettal, S., Langmead, C. & Russell, A. AI can help to speed up drug discovery — but only if we give it the right data. Nature 621 , 467–470 (2023).

Qureshi, R. et al. AI in drug discovery and its clinical relevance. Heliyon 9 , e17575 (2023).

Pati, S. et al. Federated learning enables big data for rare cancer boundary detection. Nat. Commun. 13 , 7346 (2022).

Schork, N. J. Artificial intelligence and personalized medicine. Cancer Treat. Res. 178 , 265 (2019).

Uddin, M., Wang, Y. & Woodbury-Smith, M. Artificial intelligence for precision medicine in neurodevelopmental disorders. npj Digit. Med. 2 , 1–10 (2019).

Hashimoto, D. A., Rosman, G., Rus, D. & Meireles, O. R. Artificial intelligence in surgery: promises and perils. Ann. Surg. 268 , 70–76 (2018).

Article   PubMed   Google Scholar  

Chen, R. J. et al. Algorithmic fairness in artificial intelligence for medicine and healthcare. Nat. Biomed. Eng. 7 , 719–742 (2023).

Thomasian, N. M., Eickhoff, C. & Adashi, E. Y. Advancing health equity with artificial intelligence. J. Public Health Policy 42 , 602–611 (2021).

Zheng, Y., Carrillo-Perez, F., Pizurica, M., Heiland, D. H. & Gevaert, O. Spatial cellular architecture predicts prognosis in glioblastoma. Nat. Commun. 14 , 4122 (2023).

Wu, J. et al. Radiological tumour classification across imaging modality and histology. Nat. Mach. Intell. 3 , 787–798 (2021).

García-Figueiras, R. et al. Proton magnetic resonance spectroscopy in oncology: the fingerprints of cancer? Diagn. Interv. Radiol . 22 , 75–89 (2016).

Phillips, H. S. et al. Molecular subclasses of high-grade glioma predict prognosis, delineate a pattern of disease progression, and resemble stages in neurogenesis. Cancer Cell 9 , 157–173 (2006).

Article   CAS   PubMed   Google Scholar  

Louis, D. N. et al. The 2021 WHO classification of tumors of the central nervous system: a summary. Neuro. Oncol. 23 , 1231–1251 (2021).

Alix-Panabières, C. & Pantel, K. Liquid biopsy: from discovery to clinical application. Cancer Discov. 11 , 858–873 (2021).

Yang, H. et al. Cerebrospinal fluid-derived circulating tumor DNA is more comprehensive than plasma in NSCLC patients with leptomeningeal metastases regardless of extracranial evolution. Heliyon 8 , e12374 (2022).

Lehner, K. R., Jiang, K., Rincon-Torroella, J., Perera, R. & Bettegowda, C. Cerebrospinal fluid biomarkers in pediatric brain tumors: a systematic review. Neoplasia 35 , 100852 (2023).

Liu, H. et al. M3AE: multimodal representation learning for brain tumor segmentation with missing modalities. Proc. AAAI Conf. Artif. Intell. 37 , 1657–1665 (2023).

Google Scholar  

Clark, K. et al. The cancer imaging archive (TCIA): maintaining and operating a public information repository. J. Digit. Imaging 26 , 1045–1057 (2013).

Menze, B. H. et al. The multimodal brain tumor image segmentation benchmark (BRATS). IEEE Trans. Med. Imaging 34 , 1993–2024 (2015).

Therapeutically applicable research to generate effective treatments (TARGET) - NCI. https://www.cancer.gov/ccg/research/genome-sequencing/target .

GDC. https://portal.gdc.cancer.gov/ .

Welcome to the cancer imaging archive - the cancer imaging archive (TCIA). https://www.cancerimagingarchive.net/ .

Xia, M., Wang, J. & He, Y. BrainNet viewer: a network visualization tool for human brain connectomics. PLoS One 8 , e68910 (2013).

Sun, L., Zhang, S., Chen, H. & Luo, L. Brain tumor segmentation and survival prediction using multimodal MRI scans with deep learning. Front. Neurosci. 13 , 810 (2019).

Feng, X., Tustison, N. J., Patel, S. H. & Meyer, C. H. Brain tumor segmentation using an ensemble of 3D U-nets and overall survival prediction using radiomic features. Front. Comput. Neurosci. 14 , 25 (2020).

ZainEldin, H. et al. Brain tumor detection and classification using deep learning and sine-cosine fitness grey wolf optimization. Bioengineering 10 , 1–19 (2023).

Latif, G., Iskandar, D. N. F. A., Alghazo, J. & Butt, M. M. Brain MR image classification for glioma tumor detection using deep convolutional neural network features. Curr. Med. imaging 17 , 56–63 (2021).

CAS   PubMed   Google Scholar  

Saeedi, S., Rezayi, S., Keshavarz, H. & R. Niakan Kalhori, S. MRI-based brain tumor detection using convolutional deep learning methods and chosen machine learning techniques. BMC Med. Inform. Decis. Mak. 23 , 16 (2023).

Bhandari, A., Koppen, J. & Agzarian, M. Convolutional neural networks for brain tumour segmentation. Insights Imaging 11 , 77 (2020).

Chen, S., Ding, C. & Liu, M. Dual-force convolutional neural networks for accurate brain tumor segmentation. Pattern Recognit . 88 , 90–100 (2019).

Isensee, F., Jaeger, P. F., Kohl, S. A. A., Petersen, J. & Maier-Hein, K. H. nnU-Net: a self-configuring method for deep learning-based biomedical image segmentation. Nat. Methods 18 , 203–211 (2021).

Steyaert, S. et al. Multimodal deep learning to predict prognosis in adult and pediatric brain tumors. Commun. Med. 3 , 44 (2023).

Kamnitsas, K. et al. Efficient multi-scale 3D CNN with fully connected CRF for accurate brain lesion segmentation. Med. Image Anal. 36 , 61–78 (2017).

Milletari, F., Navab, N. & Ahmadi, S. A. V-Net: fully convolutional neural networks for volumetric medical image segmentation. In : 2016 Fourth International Conference on 3D Vision (3DV), Stanford, CA, USA, 565–571 (2016).

https://www.cancer.org/cancer/types/brain-spinal-cord-tumors-adults/detection-diagnosis-staging/how-diagnosed.html Tests for Brain and Spinal Cord Tumors in Adults.

Nasrallah, M. P. et al. Molecular neuropathology in practice: clinical profiling and integrative analysis of molecular alterations in glioblastoma. Acad. Pathol . 6 , 2374289519848353 (2019).

Afridi, M., Jain, A., Aboian, M. & Payabvash, S. Brain tumor imaging: applications of artificial intelligence. Semin. Ultrasound Ct. MR 43 , 153–169 (2022).

Ellingson, B. M., Wen, P. Y., Van Den Bent, M. J. & Cloughesy, T. F. Pros and cons of current brain tumor imaging. Neuro. Oncol. 16 , vii2 (2014).

Ghandour, F. et al. Presenting psychiatric and neurological symptoms and signs of brain tumors before diagnosis: a systematic review. Brain Sci. 11 , 1–20 (2021).

Grant, R. et al. Interventions to reduce the time to diagnosis of brain tumours. Cochrane Database Syst. Rev. 9 , CD013564 (2020).

PubMed   Google Scholar  

Iijima, K. et al. Microrecording and image-guided stereotactic biopsy of deep-seated brain tumors. J. Neurosurg. 123 , 978–988 (2015).

Luo, Q., Li, Y., Luo, L. & Diao, W. Comparisons of the accuracy of radiation diagnostic modalities in brain tumor: a nonrandomized, nonexperimental, cross-sectional trial. Med. 97 , e11256 (2018).

Histed, S. N. et al. Review of functional/ anatomic imaging in oncology. Nucl. Med. Commun. 33 , 349 (2012).

Riche, M. et al. Complications after frame-based stereotactic brain biopsy: a systematic review. Neurosurg. Rev. 44 , 301–307 (2021).

Keane, L., Cheray, M., Blomgren, K. & Joseph, B. Multifaceted microglia - key players in primary brain tumour heterogeneity. Nat. Rev. Neurol. 17 , 243–259 (2021).

Martucci, M. et al. Magnetic resonance imaging of primary adult brain tumors: state of the art and future perspectives. Biomedicines 11 , 364 (2023).

Zhang, B., Shi, H. & Wang, H. Machine learning and AI in cancer prognosis, prediction, and treatment selection: a critical approach. J. Multidiscip. Healthc. 16 , 1779–1791 (2023).

Bauer, A. H., Erly, W., Moser, F. G., Maya, M. & Nael, K. Differentiation of solitary brain metastasis from glioblastoma multiforme: a predictive multiparametric approach using combined MR diffusion and perfusion. Neuroradiology 57 , 697–703 (2015).

Voicu, I. P. et al. Differentiating solitary brain metastases from high-grade gliomas with MR: comparing qualitative versus quantitative diagnostic strategies. Radiol. Med. 127 , 891–898 (2022).

Kunimatsu, A. et al. Texture analysis in brain tumor MR imaging. Magn. Reson. Med. Sci. 21 , 95–109 (2022).

Soni, X. N., Priya, S. & Bathla, X. G. Texture analysis in cerebral gliomas: a review of the literature. AJNR Am. J. Neuroradiol. 40 , 928 (2019).

Bharath, K., Kurtek, S., Rao, A. & Baladandayuthapani, V. Radiologic image-based statistical shape analysis of brain tumours. J. R. Stat. Soc. Ser. C. Appl. Stat. 67 , 1357–1378 (2018).

Rajan, P. G. & Sundar, C. Brain tumor detection and segmentation by intensity adjustment. J. Med. Syst. 43 , 282 (2019).

Kader et al. Brain tumor detection and classification on MR images by a deep wavelet auto-encoder model. Diagnostics 11 , 1589 (2021).

Nie, D. et al. Multi-channel 3D deep feature learning for survival time prediction of brain tumor patients using multi-modal neuroimages. Sci. Rep. 9 , 1–14 (2019).

Srinivas, C. et al. Deep transfer learning approaches in performance analysis of brain tumor classification using MRI images. J. Healthc. Eng. 2022 , 3264367 (2022).

Ali, H. et al. The role of generative adversarial networks in brain MRI: a scoping review. Insights Imaging 13 , 98 (2022).

Guo, W. et al. Multiparametric MRI-based radiomics model for predicting H3 K27M mutant status in diffuse midline glioma: a comparative study across different sequences and machine learning techniques. Front. Oncol. 12 , 796583 (2022).

Kihira, S. et al. Multiparametric MRI texture analysis in prediction of glioma biomarker status: added value of MR diffusion. Neuro-Oncol. Adv. 3 , vdab051 (2021).

Schepke, E. et al. DNA methylation profiling improves routine diagnosis of paediatric central nervous system tumours: a prospective population-based study. Neuropathol. Appl. Neurobiol. 48 , e12838 (2022).

Hollon, T. et al. Artificial-intelligence-based molecular classification of diffuse gliomas using rapid, label-free optical imaging. Nat. Med. 29 , 828–832 (2023).

Kim, M. et al. Diffusion- and perfusion-weighted MRI radiomics model may predict isocitrate dehydrogenase (IDH) mutation and tumor aggressiveness in diffuse lower grade glioma. Eur. Radiol. 30 , 2142–2151 (2020).

Sun, Z. et al. Prediction of IDH mutation status of glioma based on terahertz spectral data. Spectrochim. Acta A Mol. Biomol. Spectrosc. 295 , 122629 (2023).

Hajri, R., Nicod-Lalonde, M., Hottinger, A. F., Prior, J. O. & Dunet, V. Prediction of glioma grade and IDH status using 18F-FET PET/CT dynamic and multiparametric texture analysis. Diagnostics 13 , 2604 (2023).

Yan, J. et al. Predicting 1p/19q co-deletion status from magnetic resonance imaging using deep learning in adult-type diffuse lower-grade gliomas: a discovery and validation study. Lab. Investig. 102 , 154–159 (2022).

Murdaugh, R. L. & Anastas, J. N. Applying single cell multi-omic analyses to understand treatment resistance in pediatric high grade glioma. Front. Pharmacol. 14 , 1002296 (2023).

Kool, M. et al. Integrated genomics identifies five medulloblastoma subtypes with distinct genetic profiles, pathway signatures and clinicopathological features. PLoS One 3 , e3088 (2008).

Bender, K. et al. High-grade astrocytoma with piloid features (HGAP): the Charité experience with a new central nervous system tumor entity. J. Neurooncol. 153 , 109–120 (2021).

Vermeulen, C. et al. Ultra-fast deep-learned CNS tumour classification during surgery. Neuro. Oncol . 622 , 842–849 (2023).

Rees, J. H. Diagnosis and treatment in neuro-oncology: an oncological perspective. Br. J. Radiol. 84 , S82–S89 (2011).

Mariotto, A. B. et al. Cancer survival: an overview of measures, uses, and interpretation. J. Natl Cancer Inst. Monogr. 2014 , 145–186 (2014).

Kickingereder, P. et al. Radiomic profiling of glioblastoma: identifying an imaging predictor of patient survival with improved performance over established clinical and radiologic risk models. Radiology 280 , 880–889 (2016).

Prasanna, P., Patel, J., Partovi, S., Madabhushi, A. & Tiwari, P. Radiomic features from the peritumoral brain parenchyma on treatment-naïve multi-parametric MR imaging predict long versus short-term survival in glioblastoma multiforme: preliminary findings. Eur. Radiol. 27 , 4198–4199 (2017).

Kickingereder, P. et al. Radiomic subtyping improves disease stratification beyond key molecular, clinical, and standard imaging characteristics in patients with glioblastoma. Neuro. Oncol. 20 , 848–857 (2018).

Kim, J. Y. et al. Radiomics in peritumoral non-enhancing regions: fractional anisotropy and cerebral blood volume improve prediction of local progression and overall survival in patients with glioblastoma. Neuroradiology 61 , 1261–1272 (2019).

Li, G. et al. An MRI radiomics approach to predict survival and tumour-infiltrating macrophages in gliomas. Brain 145 , 1151–1161 (2022).

Iyer, S. et al. Novel MRI deformation-heterogeneity radiomic features are associated with molecular subgroups and overall survival in pediatric medulloblastoma: preliminary findings from a multi-institutional study. Front. Oncol. 12 , 915143 (2022).

Long, H. et al. MRI radiomic features of peritumoral edema may predict the recurrence sites of glioblastoma multiforme. Front. Oncol . 12 , 1042498 (2023).

Zhou, T. et al. Prediction of brain tumor recurrence location based on multi-modal fusion and nonlinear correlation learning. Comput. Med. Imaging Graph. 106 , 102218 (2023).

Śledzińska, P., Bebyn, M. G., Furtak, J., Kowalewski, J. & Lewandowska, M. A. Prognostic and predictive biomarkers in gliomas. Int. J. Mol. Sci . 22 , 10373 (2021).

Wang, C., Zhu, X., Hong, J. C. & Zheng, D. Artificial intelligence in radiotherapy treatment planning: present and future. Technol. Cancer Res. Treat . 18 , https://doi.org/10.1177/1533033819873922 (2019).

Creasy, J. M. et al. Quantitative imaging features of pretreatment CT predict volumetric response to chemotherapy in patients with colorectal liver metastases. Eur. Radiol. 29 , 458–467 (2019).

Kawahara, D., Tang, X., Lee, C. K., Nagata, Y. & Watanabe, Y. Predicting the local response of metastatic brain tumor to gamma knife radiosurgery by radiomics with a machine learning method. Front. Oncol . 10 , 569461 (2021).

Wang, Y. et al. The radiomic-clinical model using the SHAP method for assessing the treatment response of whole-brain radiotherapy: a multicentric study. Eur. Radiol. 32 , 8737–8747 (2022).

Yang, Y. et al. Spatial heterogeneity of edema region uncovers survival-relevant habitat of Glioblastoma. Eur. J. Radiol . 154 , 110423 (2022).

Do, D. T., Yang, M. R., Lam, L. H. T., Le, N. Q. K. & Wu, Y. W. Improving MGMT methylation status prediction of glioblastoma through optimizing radiomics features using genetic algorithm-based machine learning approach. Sci. Rep . 12 , 13412 (2022).

Boehm, K. M. & Khosravi, P. Harnessing multimodal data integration to advance precision oncology. 22 , 114–126 (2022).

Cè, M. et al. Artificial intelligence in brain tumor imaging: a step toward personalized medicine. Curr. Oncol. 30 , 2673–2701 (2023).

Midya, A., Chakraborty, J., Gönen, M., Do, R. K. G. & Simpson, A. L. Influence of CT acquisition and reconstruction parameters on radiomic feature reproducibility. J. Med. Imaging 5 , 011020 (2018).

Zwanenburg, A. Radiomics in nuclear medicine: robustness, reproducibility, standardization, and how to avoid data analysis traps and replication crisis. Eur. J. Nucl. Med. Mol. Imaging 46 , 2638–2655 (2019).

Lambin, P. et al. Radiomics: the bridge between medical imaging and personalized medicine. Nat. Rev. Clin. Oncol . 14 , 749–762 (2017).

Park, J. E. et al. A systematic review reporting quality of radiomics research in neuro-oncology: toward clinical utility and quality improvement using high-dimensional imaging features. BMC Cancer 20 , 29 (2020).

Ambe, S. et al. Racial disparities in malignant primary brain tumor survival in Texas from 1995 to 2013. Cureus 12 , e11710 (2020)

Butterfield, J. T. et al. Racial disparities in recommendations for surgical resection of primary brain tumours: a registry-based cohort analysis. Lancet 400 , 2063–2073 (2022).

Carrano, A., Juarez, J. J., Incontri, D., Ibarra, A. & Cazares, H. G. Sex-specific differences in glioblastoma. Cells 10 , 1783 (2021).

Shreve, J. T., Khanani, S. A. & Haddad, T. C. Artificial intelligence in oncology: current capabilities, future opportunities, and ethical considerations. Am. Soc. Clin. Oncol. Educ. B . 42 , 1–10 (2022)

Naik, N. et al. Legal and ethical consideration in artificial intelligence in healthcare: who takes responsibility? Front. Surg. 9 , 862322 (2022).

Amann, J., Blasimme, A., Vayena, E., Frey, D. & Madai, V. I. Explainability for artificial intelligence in healthcare: a multidisciplinary perspective. BMC Med. Inform. Decis. Mak . 20 , 310 (2020).

Holzinger, A., Langs, G., Denk, H., Zatloukal, K. & Müller, H. Causability and explainability of artificial intelligence in medicine. Wiley Interdiscip. Rev. Data Min. Knowl. Discov. 9 , e1312 (2019).

DeYoe, E. A., Bandettini, P., Neitz, J., Miller, D. & Winans, P. Functional magnetic resonance imaging (FMRI) of the human brain. J. Neurosci. Methods 54 , 171–187 (1994).

Assaf, Y. & Pasternak, O. Diffusion tensor imaging (DTI)-based white matter mapping in brain research: a review. J. Mol. Neurosci. 34 , 51–61 (2008).

Singh, N. M. et al. Data consistent deep rigid MRI motion correction. https://arxiv.org/abs/2301.10365 (2023).

Chen, Z. et al. Deep learning for image enhancement and correction in magnetic resonance imaging—state-of-the-art and challenges. J. Digit. Imaging 36 , 204 (2023).

Han, K. et al. A survey on vision transformer. IEEE Trans. Pattern Anal. Mach. Intell . 45 , 87–110 (2022).

Asiri, A. A. et al. Exploring the power of deep learning: fine-tuned vision transformer for accurate and efficient brain tumor detection in MRI scans. Diagnostics 13 , 2094 (2023).

Huang, L. et al. A transformer-based generative adversarial network for brain tumor segmentation. Front. Neurosci . 16 , 1054948 (2022).

Saueressig, C., Berkley, A., Kang, E., Munbodh, R. & Singh, R. Exploring graph-based neural networks for automatic brain tumor segmentation. Lect. Notes Comput. Sci. 12611 , 18–37 (2021).

Ravinder, M. et al. Enhanced brain tumor classification using graph convolutional neural network architecture. Sci. Rep. 13 , 1–22 (2023).

Zeineldin, R. A. et al. Explainability of deep neural networks for MRI analysis of brain tumors. Int. J. Comput. Assist. Radiol. Surg . 17 , 1673–1683 (2022).

Esmaeili, M., Vettukattil, R., Banitalebi, H., Krogh, N. R. & Geitung, J. T. Explainable artificial intelligence for human-machine interaction in brain tumor localization. J. Pers. Med. 11 , 1213 (2021).

Ogier du Terrail, J. et al. Federated learning for predicting histological response to neoadjuvant chemotherapy in triple-negative breast cancer. Nat. Med. 29 , 135–146 (2023).

Nasrallah, M. P. et al. Machine learning for cryosection pathology predicts the 2021 WHO classification of glioma. Med 4 , 526-540.e4 (2023).

Romano, M. F., Shih, L. C., Paschalidis, I. C., Au, R. & Kolachalama, V. B. Large language models in neurology research and future practice. Neurology 101 , 1058–1067 (2023).

Download references

Acknowledgements

Research reported in this publication was supported by Emory Radiology-Pathology-Informatics Seed Grant Award Project 00128909, and Project 00131546 (M.A), the National Cancer Institute under award numbers R01CA249992-01A1, R01CA202752-01A1, R01CA208236-01A1, R01CA216579-01A1, R01CA220581-01A1, R01CA257612-01A1, 1U01CA239055-01, 1U01CA248226-01, 1U54CA254566-01, National Heart, Lung and Blood Institute 1R01HL15127701A1, R01HL15807101A1, National Institute of Biomedical Imaging and Bioengineering 1R43EB028736-01, National Center for Research Resources under award number 1 C06 RR12463-01, VA Merit Review Award IBX004121A (A.M) from the United States Department of Veterans Affairs Biomedical Laboratory Research and Development Service the Office of the Assistant Secretary of Defense for Health Affairs, through the Breast Cancer Research Program (W81XWH-19-1-0668), the Prostate Cancer Research Program (W81XWH-15-1-0558, W81XWH-20-1-0851), the Lung Cancer Research Program (W81XWH-18-1-0440, W81XWH-20-1-0595), the Peer-Reviewed Cancer Research Program (W81XWH-18-1-0404, W81XWH-21-1-0345, W81XWH-21-1-0160), the Mayo Clinic Breast Cancer SPORE grant P50 CA116201 from the NIH, the Kidney Precision Medicine Project (KPMP) Glue Grant, and sponsored research agreements from Bristol Myers-Squibb, Boehringer-Ingelheim, Eli-Lilly and AstraZeneca. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health, the U.S. Department of Veterans Affairs, the Department of Defense, or the United States Government.

Author information

Authors and affiliations.

Wallace H. Coulter Department of Biomedical Engineering, Georgia Institute of Technology and Emory University, Atlanta, GA, USA

Sirvan Khalighi, Abhishek Midya, Krunal Balvantbhai Pandav & Anant Madabhushi

Department of Radiology, Emory University, Atlanta, GA, USA

Kartik Reddy

Atlanta Veterans Administration Medical Center, Atlanta, GA, USA

Anant Madabhushi

Department of Pathology and Laboratory Medicine, Emory University, Atlanta, GA, USA

Malak Abedalthagafi

The Cell and Molecular Biology Program, Winship Cancer Institute, Atlanta, GA, USA

You can also search for this author in PubMed   Google Scholar

Contributions

M.A. and A.M. were responsible for the concept proposal and study design. M.A. conceptualized the study, attained funding, provided supervision, prepared the initial draft, and reviewed and revised the final manuscript. S.K. wrote the manuscript with feedback and contributions from all other authors. K.R., Ab.M. and K.B.P. contributed to the data analysis and writing of the manuscript. All authors have revised, read, and agreed to the final version of the manuscript.

Corresponding authors

Correspondence to Anant Madabhushi or Malak Abedalthagafi .

Ethics declarations

Competing interests.

Dr. Abedalthagafi is part of the Editorial Board of this journal. Dr. Madabhushi is an equity holder in Picture Health, Elucid Bioimaging, and Inspirata Inc. Currently, he serves on the advisory board of Picture Health, Aiforia Inc., and SimBioSys. He also currently consults for SimBioSys. He also has sponsored research agreements with AstraZeneca, Boehringer-Ingelheim, Eli-Lilly, and Bristol Myers-Squibb. His technology has been licensed to Picture Health and Elucid Bioimaging. He is also involved in 3 different R01 grants with Inspirata Inc. He also serves as a member of the Frederick National Laboratory Advisory Committee. The remaining authors declare no competing interests.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary information

Supp tables, rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ .

Reprints and permissions

About this article

Cite this article.

Khalighi, S., Reddy, K., Midya, A. et al. Artificial intelligence in neuro-oncology: advances and challenges in brain tumor diagnosis, prognosis, and precision treatment. npj Precis. Onc. 8 , 80 (2024). https://doi.org/10.1038/s41698-024-00575-0

Download citation

Received : 19 October 2023

Accepted : 13 March 2024

Published : 29 March 2024

DOI : https://doi.org/10.1038/s41698-024-00575-0

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

Quick links

• Explore articles by subject
• Guide to authors
• Editorial policies

Sign up for the Nature Briefing: Cancer newsletter — what matters in cancer research, free to your inbox weekly.

International Joint Conference on Advances in Computational Intelligence

IJCACI 2022: Proceedings of International Joint Conference on Advances in Computational Intelligence pp 449–461 Cite as

Brain Tumor Detection Using Convolutional Neural Network

• Vijay Mane 6 ,
• Amay Chivate 6 ,
• Prajyot Ambekar 6 ,
• Ananya Chavan 6 &
• Ameya Pangavhane 6  
• Conference paper
• First Online: 02 April 2024

Part of the book series: Algorithms for Intelligent Systems ((AIS))

Brain tumors pose a substantial risk to one's health, and early detection is necessary for successful treatment and patient outcomes. Convolutional neural networks (CNNs), in particular, are currently demonstrating astounding performance in a number of health-related imaging tasks, such as the detection of tumors in the brain. This study presents an original way of locating brain tumors using CNNs. The proposed method utilizes an assortment of brain MRI scans, consisting of both cancer and non-cancer samples, for training the CNN model. The CNN's structure is intended to effectively grasp spatial patterns and features within the images. To enhance the unit's functionality, several data preprocessing tactics, like normalization and augmentation, are employed. Additionally, transferring knowledge may be leveraged beginning with the CNN with acquired weights obtained from a large-scale image recognition task. The trained CNN model is assessed using separate sample data, and its performance is compared to other modern methods of technology methods. The outcomes show the effectiveness of the suggested approach, achieving high precision, sensitivity, and clarity in brain tumor detection. The study's conclusions highlight the CNNs' promise as an asset in order to detect malignancies in the brain efficiently. On a test dataset of brain MRI pictures, the suggested method yielded a loss of 0.1314 and an accuracy of 0.9420. Further research can focus on incorporating additional modalities and expanding the application using deep computing in other aspects of brain cancer analysis, such as segmentation and classification of tumor subtypes.

• Medical image
• Segmentation

This is a preview of subscription content, log in via an institution .

Buying options

• Available as PDF
• Read on any device
• Instant download
• Own it forever
• Available as EPUB and PDF
• Durable hardcover edition
• Dispatched in 3 to 5 business days
• Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Hemanth G, Janardhan M, Sujihelen L (2019) Design and implement brain tumor detection using machine learning approach. In: 2019 3rd international conference on trends in Electronics and informatics (ICOEI). IEEE, pp 1289–1294

Google Scholar  

Seetha J, Raja SS (2018) Brain tumor classification using convolutional neural networks. Biomed Pharmacol J 11(3):1457

Article   Google Scholar  

Myronenko A (2019) 3D MRI brain tumor segmentation using autoencoder regularization. In: Brainlesion: Glioma, multiple sclerosis, stroke and traumatic brain injuries: 4th international workshop, BrainLes 2018, held in conjunction with MICCAI 2018, Granada, Spain, September 16, 2018, Revised Selected Papers, Part II 4. Springer International Publishing, pp 311–320

Malathi M, Sinthia P (2019) Brain tumour segmentation using convolutional neural network with tensor flow. Asian Pac J Cancer Prev APJCP 20(7):2095

Khan MSI, Rahman A, Debnath T, Karim MR, Nasir MK, Band SS, Dehzangi I et al. (2022) Accurate brain tumor detection using deep convolutional neural network. Comput Struct Biotechnol J 20:4733–4745

Amin J, Sharif M, Haldorai A, Yasmin M, Nayak RS (2021) Brain tumor detection and classification using machine learning: a comprehensive survey. Complex Intell Syst 1–23

Wang G, Li W, Ourselin S, Vercauteren T (2018) Automatic brain tumor segmentation using cascaded anisotropic convolutional neural networks. In: Brainlesion: glioma, multiple sclerosis, stroke and traumatic brain injuries: third international workshop, BrainLes 2017, Held in Conjunction with MICCAI 2017, Quebec City, QC, Canada, September 14, 2017, Revised Selected Papers 3. Springer International Publishing, pp 178–190

Isensee F, Kickingereder P, Wick W, Bendszus M, Maier-Hein KH (2018) Brain tumor segmentation and radionics survival prediction: Contribution to the brats 2017 challenge. In: Brainlesion: glioma, multiple sclerosis, stroke and traumatic brain injuries: third international workshop, BrainLes 2017, Held in Conjunction with MICCAI 2017, Quebec City, QC, Canada, September 14, 2017, Revised Selected Papers 3. Springer International Publishing, pp 287–297

Ranjbarzadeh R, Bagherian Kasgari A, Jafarzadeh Ghoushchi S, Anari S, Naseri M, Bendechache M (2021) Brain tumor segmentation based on deep learning and an attention mechanism using MRI multi-modalities brain images. Sci Rep 11(1):10930

Brindha PG, Kavinraj M, Manivasakam P, Prasanth P (2021) Brain tumor detection from MRI images using deep learning techniques. In: IOP conference series: materials science and engineering, vol. 1055, No. 1. IOP Publishing, p 012115

Chattopadhyay A, Maitra M (2022) MRI-based brain tumour image detection using CNN based deep learning method. Neurosci Inf 2(4):100060

Naveen V, Gokul V, Thilagavathi G, Harshavardhan PR, Kumar N (2023) Deep learning technique to detect and classify brain tumor. In: 2023 9th international conference on advanced computing and communication systems (ICACCS), vol 1. IEEE, pp 699–703

Hossain T, Shishir FS, Ashraf M, Al Nasim MA, Shah FM (2019) Brain tumor detection using convolutional neural network. In: 2019 1st international conference on advances in science, engineering and robotics technology (ICASERT). IEEE, pp 1–6

Febrianto DC, Soesanti I, Nugroho HA (2020) Convolutional neural network for brain tumor detection. In: IOP conference series: materials science and engineering, vol. 771, No. 1, p. 012031). IOP Publishing, p 012031

Msoud Nickparvar (2021).Brain tumor MRI dataset [Dataset] Kaggle. https://doi.org/10.34740/KAGGLE/DSV/2645886

Raja SG, Nirmala K (2019) Detection of brain tumor using K-nearest neighbor (KNN) based classification model and self-organizing map (SOM) algorithm. Int J Innovative Technol Explor Eng (IJITEE) 8(8)

Download references

Author information

Authors and affiliations.

Department of Electronics and Telecommunication Engineering, Vishwakarma Institute of Technology, Pune, Maharashtra, 411037, India

Vijay Mane, Amay Chivate, Prajyot Ambekar, Ananya Chavan & Ameya Pangavhane

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Amay Chivate .

Editor information

Editors and affiliations.

Department of Computer Science and Engineering, Jahangirnagar University, Dhaka, Bangladesh

Mohammad Shorif Uddin

Department of Mathematics, South Asian University, New Delhi, Delhi, India

Jagdish Chand Bansal

Rights and permissions

Reprints and permissions

Copyright information

© 2024 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.

About this paper

Cite this paper.

Mane, V., Chivate, A., Ambekar, P., Chavan, A., Pangavhane, A. (2024). Brain Tumor Detection Using Convolutional Neural Network. In: Uddin, M.S., Bansal, J.C. (eds) Proceedings of International Joint Conference on Advances in Computational Intelligence. IJCACI 2022. Algorithms for Intelligent Systems. Springer, Singapore. https://doi.org/10.1007/978-981-97-0180-3_35

Download citation

DOI : https://doi.org/10.1007/978-981-97-0180-3_35

Published : 02 April 2024

Publisher Name : Springer, Singapore

Print ISBN : 978-981-97-0179-7

Online ISBN : 978-981-97-0180-3

eBook Packages : Intelligent Technologies and Robotics Intelligent Technologies and Robotics (R0)

Share this paper

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Publish with us

Policies and ethics

• Find a journal
• Track your research