Intrusion Detection System using Apache Spark on Hadoop Big Data Environment
Faculty Mentor
Dr. Hayden Wimmer
Location
Poster 228
Session Format
Poster Presentation
Academic Unit
Department of Information Technology
Keywords
Allen E. Paulson College of Engineering and Computing Student Research Symposium, Intrusion Detection System, IDS, Convolution Neural Network, CNN, Long-Short Term Memory, LSTM, Synthetic Minority Oversampling Technique, SMOTE, Resilient Data Distributed Datasets, RDD
Creative Commons License
This work is licensed under a Creative Commons Attribution 4.0 License.
Presentation Type and Release Option
Presentation (File Not Available for Download)
Start Date
2022 12:00 AM
January 2022
Intrusion Detection System using Apache Spark on Hadoop Big Data Environment
Poster 228
Intrusion Detection System (IDS) is a system that uses intelligent techniques to analyze data quickly and capture the attacks at an early stage. Based on detection methods IDS can be Signature-based, this is an IDS that detects known attacks ,the major limitation is that this detection system cannot detect unknown attacks or Anomaly-based that detects attacks based on any abnormality and is able to detect unknown attacks hence overcome the drawbacks of signature-based methods. Spark is distributed processing system used for big data that utilizes in-memory caching and optimized query execution for fast queries against data of any size. Spark was designed to be fast for iterative algorithms, support for in-memory storage and efficient fault recovery. Hadoop is a big data environment, It has a distributed file system HDFS used to store data of various formats across a cluster . The purpose of the study is to build an Anomaly-based IDS using Apache Spark on Hadoop, the framework will overcome the challenges posed by traditional techniques of computational power and storage.
