Intrusion Detection Systems Using Machine

Essay, Pages 14 (3428 words)

Views

Abstract

The various machine learning algorithms work on different classification problems. One such area is intrusion detection in computer network. The computer security is the crucial area in the digital area. When datasize increases, the performance of machine learning is somewhat less than Deep Learning approach. To handle this situation Deep learning is the best solution for multi-class classification problems. Therefore this paper consists of thorough literature review on different algorithms in Machine learning and Deep learning. Various performance matrices are compared along with their attributes considered in the experiment.

Don't use plagiarized sources. Get your custom paper on

“ Intrusion Detection Systems Using Machine ”

Get high-quality paper

NEW! smart matching with writer

The aim of this paper to provide literature review on IDS system with ML and DL algorithms. At last how the Deep learning is came with its features over ML is concluded.

Introduction

In the current internet era, many security aspects are being generated. Areas such as education, banking, research, mil- itary, space, government etc. with reference to this various threats are being penetrated by attackers for many purposes that affects both computer and computer network.

As the no system in the world is 100% secure. There are at least vulner abilities available in the computer system. Hence information security comes into picture. Many attacks are happening now and then; the most happened attacks are DoS, XSS, SQL injection, HTTP response splitting etc. There are various methods available to detect these attacks such as signature based, ML, DL etc. As the DL has some of the advantages over ML for specific kind of attacks. Both ML and DL can detect new attacks but DL has some of the advantages such as it can handle large amount of data and can classify even if classification classes are much more.

Literature Review

The [1] proposed technique called Network Anomaly De- tection utilizing Active Learning (NADAL) contains the two essential advances i.e. offline and online to handle the issues of spilling nature of the information. Toward the starting the NSL-KDD dataset is utilized and preprocessed in disconnected advance which has assaults called DoS, Probe, R2L, and U2R. The dataset is a modification of the KDD-99 without tedious and repetitive cases and 42 features. The dataset was randomized by means of the Randomize usefulness in Weka. The KDD train+.txt document was utilized wherein the 42nd features recognizes an normal versus attack mark.

The chose features are then given to the element sifting module in NADAL. At each layer, the naive Bayesian module incre- mentally predicts the likelihood that the occasion has a place with the class. The yield of the system decides if the mark for the example must be asked. The dynamic learning modules and in addition the incremental naive Bayesian module were actualized by changing the code from Massive Online Analysis (MOA1) 2016.04 written in Java. The accuracy and Kappa parameters were then figured for the structure at four layers: DoS, Probe, U2R, and R2L. The outcomes were contrasted with those of the incremental credulous Bayesian approach in MOA. Besides, regarding R2L and U2R, the proposed technique has higher exactness by and large.

In [2], proposed a productive BPNN design for the ad- vancement of an irregularity based IDS with high accuracy and detection rate. The KDD’99 informational collection is utilized as a part of this setting to build up the design and utilized its 20% to prepare the system. In the proposed system, 40 features are chosen rather than 41 in light of the fact that 1 include has every one of the 0 as value. Every one of the 0 esteems don’t influence any adjustment in interruption accuracy rates and subsequently removed. The experiment is carried out using different number of hidden layers with different combinations of hidden nodes. After the examination they showed signs of improvement location rate and precision with 4 hidden layers and certain combinations of hidden nodes.

The experimentation is finished utilizing two techniques , first by part dataset into 70% learning-40% Test and 80% learning and 20% Test and it is watched that, 80-20 % split takes more execution time than 70-30 %. With the expansion in number of nodes in each hidden layer the execution time increments definitely for both datasets. In 80-20 % split dataset when we increment the quantity of nodes in each hidden layer, the Mean Square Error (MSE) falls radically close around 0.004. Thusly, subsequent to examining every one of the outcomes, it is demonstrated that mix of 70 30% dataset is the best BPNN design.

In this paper [3], a novel detection technique is proposed in view of the self-similarity estimation of the system. The paper incorporates the novel strategy to distinguish framework and system inconsistency in view of self-similitude estimation organization of self-comparability based IDS. With a specific end goal to assess the self-similarity property, the creators have utilized two datasets for the trials: the DARPA and their own particular dataset. For the estimation of self-similarity from arrange traffics, they propelled the attacks utilizing Tenable Nessus Scanner. For that they used Denial of Service, Port Scanning, and Windows plugins in Nessus. The results of logs reveal that there is an evident self-similarity property based on the fact that the Hurst parameter ( H ) is 0.96554 and 0.98865, separately. As the Hurst parameter turns out to be more like 1 that implies level of self-similarity increments. The test comes about demonstrate that this approach has accomplished a high identification rate.

S. M. Hussein has created hybrid approach [4] for detection of attacks on Knowledge Discovery Data Mining (KDD) CUP 99 dataset and Waikato Environment for Knowledge Analysis (WEKA) program is utilized for testing. The kinds of attacks recognized are; back, satan, smurf, spy, teardrop, warezclient, perl, buffer overflow, warezmaster, multihop , Neptune, nmap,normal, phf, portsweep,pod, land, , imap, ipsweep, ftp-write, loadmodule, rootkit and guess password. The proposed approach utilizes both signature based and be- haviour based identification techniques. The two strategies for IDS have restriction to recognize known and obscure attacks notwithstanding decrease the false alert rate. At first stage grunt is utilized for examination the KDD container 99 dataset and process the whole dataset. The yield of this first stage is ready document which is created for additionally handling.

In the second stage the uninfected bundles are dissected by utilizing any of the one of Nave Bayes calculation, K-implies calculation and Bayes Net calculation utilizing as a part of WEKA program. After this, correlation is performed among every one of the outcomes created from WEKA keeping in mind the end goal to assess execution of this new Hybrid IDS. This new Hybrid approach is enhanced the execution of IDS to recognize and decrease false cautions. In a few systems, for example, assessing the accuracy to grouped class effectively, Bayes net is the best. While the aftereffect of the experiment demonstrated that the detection rate is higher when we utilized Nave base contrasted with different calculations. Ultimately the level of having false cautions represent that K- means calculation has better impact analyzed than Bayes Net and Nave Bayes individually.

This paper [5] proposes new technique which is signature based abnormality identification conspire (SADS) to investi- gate the conduct of packet header examples to limit false alert and issues identified with execution caused because of tedious calculations. SADS conspire utilizes data mining classifiers, for example, naive bayes and random forest for this reason.

The DARPA 1999 and ISCX 2012 datasets are utilized to assess execution of SADS. The test results obtained are better than existing Anomaly based detection systems. This paper [6] builds up an attack signature age approach that can be utilized for signature-based Intrusion Detection System (IDS) which depends on GA to create new attack marks from an arrangement of introductory attack signature. Attack signature generation comprises of various advances beginning with proposed approach, Use of Genetic Algorithm, Chromosome representation, Fitness function crossover and mutation. For the experimentation they have made their own dataset. The approach presently addresses the creation of both straightforward and complex type of attacks including SQL injection, XPath injection, DoS and privilege escalation. A GA needs an arrangement of population questions as input and produces another arrangement of population as output. The generated output population comprises of new and different objects as compared with original population.

Georgios P. Spathoulas et al. in [7] have worked on decreasing false positives forIDS. A post-processing filter is proposed to reduce false positives in network-based IDS. The proposed filter consists of three components, namely the Neighboring Related Alerts (NRA) component, the High Alert Frequency (HAF) component and the Usual False Positives (UFP) component. The score (belief) is calculated by each component for every alert. The scores are combined, and a final verdict is decided whether each alert is a true positive or not. Wrong alerts can be detected by the frequency with which their signature triggers false positives. Evaluation results performed using the DARPA 1999 dataset indicate that the proposed approach reduces the number of false positives with promising results. The filter limited false positives by a percentage up to 75%. The NRA component is depend on the assumption that a true positive alert must be part of a neighboring set of alerts with similarities in source or destination IPs; the HAF component is based on the observation those actual attacks.

The UFP component is the easiest of the three and tries to detect usual FPs in a network and discard them. The scores produced by the three components are then combined which is called as a general score. The number of alerts was reduced by 29%, the number of FPs was reduced by 74%, while their percentage was reduced by 63%.

This paper [8] uses novel deep learning technique for intrusion detection, the authors have proposed two techniques called non-symmetric deep auto-encoder (NDAE) and novel deep learning model developed using stacked NDAEs. NDAE is an auto-encoder which is non-symmetrical multiple hidden layers. NDAE can be used as a hierarchical unsupervised feature extractor. The reason behind its use is that to reduce both computational and time overhead along with accuracy.

The experiment has used graphics processing unit (GPU)- enabled TensorFlow, KDD Cup 99 and NSL-KDD datasets. Stacking the NDAEs offers a layer-wise unsupervised rep- resentation learning algorithm. It also has feature extraction capabilities, so it is able to refine the model by prioritizing the most descriptive features. They have combined the deep learning power of stacked NDAEs with a shallow learning classifier. In the experiments, the NSL-KDD dataset is used with the 10-fold cross-validation approach in Scikit Learn. The KDD Cup 99 Classification 5-classes dataset evaluation, the results show that proposed model gives an average accuracy of 97.85%.The proposed model has produced promising results with F-measure as 87.37%, recall is 85.42% and precision is 100.00%.The proposed model shows accuracy of 97.85%.

This paper [9] introduces a work which incorporates DM and IDS idea which is utilized for distinguishing related information with less running time. The proposed EDADT algorithm is produced for fathoming the undertaking of char- acterization of information which is Hybrid IDS. The EDADT carries on not quite the same as ordinary decision tree. To limit the workload of system administrator, SNORT is included with irregularity based methodologies. The proposed system naturally arranges the information relying upon rules inside it. The utilization of semi-regulated approach takes care of issues of directed and unsupervised techniques and semi-managed approach alludes to little measure of marked information and additionally gigantic unlabeled information can be named.

This paper [10] proposed binomial classifier in light of deep learning for NIDS. Three experiments are directed on UNSW- NB 15 dataset to decide activation function. The most critical ideal features are chosen and tried the proposed method. After pre-processing dataset, the proposed method depends on algorithm based on H2O stage which incorporates usage of multilayer feed forward ANN by utilizing back-spread. The model has five concealed layers, each layer having 10 neurons. The outcomes are tried for F-measure, accuracy, precision, Area under Curve (AUC), recall, and training time normalized to the range [0, 1]. The outcomes acquired demonstrates that the proposed display outperforms in contrast with decision tree, Logistic Regression, NB, ANN with 0.56% false alert rate and 98.99% accuracy on concealed information.

The work in [11] utilizes Back- propagation algorithm to identify attacks which are improved by Conjugate Gradient calculation (CG Optimization). The System contains the steps as pre-processing the KDD Cup 99 dataset, prepare the model with the assistance of back-spread CG algorithm. The performance matrices considered for the comparison are false alarm rate and f-measure. The framework will yield 4 class order for DoS, test, R2L, ordinary class or paired grouping for attack or normal packets. The line search technique such as golden section search, Brent search, charalambous search, and hybrid bisection search are utilized to limit learning rate performance. For line search technique hybrid bisection- cubic search provides highest average F-measure value and the lowest MSE value and turned out to be better.

The work done in [12] proposes the mixture approach by consolidating classifiers and build up a system which can settle on decisions intelligently to improve the performance. The supervised and un-supervised data filtering are used on whole training dataset and then again this output is applied to other classifiers for identification. The whole experiment is done on binary class classification i.e. intrusion and normal. The experiment is done on NSL-KDD dataset and discovered high detection rate alongside low false alarm rate. The hybrid approach is produced utilizing 10 random forest trees with 0.06 out of bag error and 100% accuracy with 0% false alarm rate, which makes the approach as the most proficient.

The authors in [13] proposed the novel method by adding the supervised, unsupervised and outlier based methods for enhance the accuracy to detect the attacks. The developed ap- proach can distinguish the new and in addition old attacks. The novel approach is the multi-level hybrid intrusion detection method. For the experiment they used their own dataset named Tezpur University intrusion detection system (TUIDS) dataset along with the use of two other datasets namely KDD Cup 99 and NSL-KDD respectively. The results are then compared for these different datasets. The last outcomes are very promising by proposed approach.

The paper [14] utilized hybrid approach by combining artificial bee colony (ABC) and AdaBoost algorithms for in- trusion detection. Using the consolidated version of these two algorithms lessened false positive rate (FPR). The selection of features is accomplished using ABC algorithm and for classification of features AdaBoost algorithm is used. The experiment is performed on two datasets named ISCXIDS2012 and NSL- KDD. The proposed approach is performing differently on various attack based datasets. The experiment is performing distinctively and giving promising results based on the attack types. The improvement n accuracy and detection rate is seen.

The results of the IDS is influenced if the data is large and un-balanced; this leads to minority class not to be properly distinguished by old data-mining algorithms. The proposed calculation additionally chooses the subset of related features to distinguish the attacks. The work in paper [15] proposed the new level hybrid attack detection model which uses the SVM and extreme learning machine to enhance the execution parameters for attack recognition. The K-means algorithm is used to give the filtered dataset and preprocess the preparation dataset to en- hance execution time. The change of the preprocessing dataset with top dataset is expert by using Kmeans calculation. After this separated dataset the location rate got made strides. The KDD Cup 99 dataset is utilized all through the examination and can identify the attack with the accuracy of 95.75% with 1.87% false alarm rate.

In [16] authors propose an effective IDS method with two step by using BC+k-NN method. The proposed approach includes two steps, first to choose the classifier among the set of classifiers along with aggregation module. Second step consists of k-NN algorithm and which works on uncertain class connections from step one and viewed as the supplemen- tary part to step one. The dataset they worked on is NL-KDD dataset. The method has detected four types of attacks namely DoS, Probe, U2R and R2L. For the detection of U2R and R2L attack the proposed method gives superior results of F1-Score as compared to baseline systems. The proposed approach gave 94.92% of accuracy, 98.72% of precision, 92.28% DR, 95.39% of F1-score and 1.59% of FAR. The results are compared with C4.5,RF, k-NN, BPNN, NB classifiers only.

In [17] authors propose the hybrid classification method using NN and Self-organizing map. The experiment is done on a NSL-KDD dataset with 41 features. The proposed method by the [17] has proved 98% of detection rate. The NN method BPNN is used for classification of four categories DoS, U2R, R2L and Probe. In the first step SOM is used for binary classification i.e. Normal or Attack. Second step is in active state if the packet is attack in first step. If the packet is classified as a normal hence no need to process the second step. The proposed method gave 96% of accuracy. The sequential approach for hybrid IDS system leads to consume more time as compared parallel hybrid IDS approach.

In [18] authors propose the Bayesian i.e. probability based IDS system to improve accuracy for R2L attack. The ex- periment is done on KDD dataset with 41 features. The different experiments have performed using these features i.e. Attack+Normal, DoS+Normal, Probe+Normal, U2R+Normal and L2R+Normal. The detection rate for R2L was 85.35%. The authors also propose to use several Bayesian filters to detect the attacks in IDS systems. The work improved the accuracy for R2L attack though it is not promising detection rate for IDS system.

In [19] the authors have used NSL-KDD dataset for IDS to detect attacks namely DoS, Probe, U2R and R2L. The random forest modeling is used for this purpose. The proposed method used various steps such as loading dataset, Preprocessing on dataset, classifying the features according to attacks, applying feature subset selection, selecting best feature subset selection, applying RF algorithm and finally to record accuracy ,DR, FAR and MCC. The results are compared for j48 and RF algorithm. RF works fine as compared to j48 algorithm in terms of Accuracy, DR, FAR and MCC. The RF performed well and showed 99.67% Accuracy, 99.84% DR, 0.00527 as FAR and 0.99 as MCC. For Probe attack it is 99.67%, 99.82%, 0.00505, 0.99 as Accuracy, DR, FAR, MCC respectively. For R2L it is 99.67%, 99.82%, 0.00505 ,0.99 as Accuracy, DR, FAR and MCC respectively. Likewise for U2R attack it is 99.67%, 99.84%, 0.00555, 0.99 as Accuracy, DR, FAR and MCC respectively. In the work evolutionary computation as a feature selection is suggested and mentioned as a future work.

In [20] authors propose the intrusion detection with the help of NB algorithm by reducing the number of features from the dataset. The three methods for reducing the number of features are investigated namely Information gain (IG), Correlation- based feature selection (CFS) and Gain Ratio (GR). The brute force approach is used to find important, less-important feature and unimportant features. The NSL-KDD dataset is used with 41 features and reduced it to 24 features using importunateness of the features. The 10-fold cross validation is used. The proposed feature reduction method called Feature Vitality based Reduction Method (FVBRM) reduced features from 41 to 24. Using the FVBRM the TPR values such as 98.70%, 98.80%, 96.10%, 64% for DoS, Probe, R2L, and U2R respectively. The proposed FVBRM improved accuracy compared to CFS but leads to more time consuming. The proposed method didnt work better for U2R attacks and has only 64% of TPR.

Methodology

The survey involved a thorough examination of research papers focusing on IDS using ML and DL. Papers were selected based on their relevance, methodology, and the datasets used. Performance metrics such as accuracy, detection rate, false positive rate, and computational efficiency served as the primary criteria for evaluation.

Results

The review revealed a discernible trend towards the adoption of DL in IDS. DL models, particularly those utilizing architectures like non-symmetric deep auto-encoders (NDAE) and stacked NDAEs, exhibited higher accuracy and detection rates compared to traditional ML models. Furthermore, DL models demonstrated enhanced capability in handling large-scale data and identifying sophisticated attack patterns.

Discussion

DL's advantages over ML in IDS can be attributed to several factors. Firstly, DL models can automatically extract and learn features from raw data, eliminating the need for manual feature engineering. Secondly, DL's inherent ability to model complex non-linear relationships allows for better generalization and detection of previously unseen attacks. However, DL models require substantial computational resources and large amounts of labeled data for training, which may pose challenges in real-world applications.

Conclusion

The transition from ML to DL in IDS marks a significant milestone in the evolution of cybersecurity technologies. While ML continues to offer value, DL's superior performance in handling complex intrusion detection tasks is undeniable. Future research should focus on optimizing DL models for real-time IDS applications and exploring unsupervised and semi-supervised DL approaches to overcome the challenges associated with labeled data scarcity.