Stacked LSTM-GRU Model for Traffic Anomalies Detection

Omar Muhammad Altoumi Alsyaibani, Ema Utami, Suwanto Raharjo, Anggit Dwi Hartanto

Abstract


This study aims to improve the accuracy of the intrusion detection system model. It focused on LSTM and GRU methods proposed by several previous studies. The bidirectional layer was also tested to see if it improves model performance. Dataset used in the study was CIC IDS 2017. The dataset was divided into 3 parts, for training, validation, and testing purposes. Validation data was used to evaluate model performance in every training iteration. It helped to make the model would not overfit the training data. Furthermore, Dropout layer and L2 regularization were also added to the model architecture. The training model was done in a binary classification approach with a learning rate of 0.0001. We found that the stacked method reached accuracy 98.1087% in 100 iteration training. This result is slightly higher than LSTM, GRU, Bidirectional LSTM, and Bidirectional GRU. The method which contains LSTM layer performed its best accuracy using Tanh activation. Differently, GRU and Bidirectional GRU performed the best performance with Lrelu and Prelu activation function, respectively. All models could reach the plateau in the first 20 iterations, while in the next 80 iterations the model performance still could be fluctuately improved. Even though the model already reached the plateau in 20 iteration training, it is still possible for the model to slowly improve by using a small learning rate and by implementing Dropout layer and L2 regularization. Fluctuation of model performance implies that the highest model performance was not always reached in the last training iteration. ModelCheckPoint could help to overcome the issue. In addition, the Bidirectional layer increased the complexity of the model which certainly increased training duration. The bidirectional layer improved the performance of the GRU method, but it did not improve the performance LSTM method.

Keywords


LSTM; GRU; IDS; ModelCheckPoint; Bidirectional Layer

Full Text:

Link Download

References


Al-Emadi, S., Al-Mohannadi, A., & Al-Senaid, F. (2020). Using Deep Learning Techniques for Network Intrusion Detection. 2020 IEEE International Conference on Informatics, IoT, and Enabling Technologies, ICIoT 2020. https://doi.org/10.1109/ICIoT48696.2020.9089524

Alsyaibani, O. M. A., Utami, E., & Hartanto, A. D. (2021). Intrusion Detection System Model Based on Gated Recurrent Unit to Detect Anomaly Traffic. 2021 4th International Conference on Information and Communications Technology (ICOIACT), 5–10. https://doi.org/10.1109/ICOIACT53268.2021.9564003

Andalib, A., & Vakili, V. T. (2020). An autonomous intrusion detection system using an ensemble of advanced learners. 2020 28th Iranian Conference on Electrical Engineering, ICEE 2020. https://doi.org/10.1109/ICEE50131.2020.9260808

Bace, R., & Mell, P. (2001). NIST special publication on intrusion detection systems. In Nist Special Publication.

Bansal, T., Belanger, D., & McCallum, A. (2016). Ask the GRU: Multi-task learning for deep text recommendations. RecSys 2016 - Proceedings of the 10th ACM Conference on Recommender Systems. https://doi.org/10.1145/2959100.2959180

Bhuvaneswari Amma, N. G., Selvakumar, S., & Leela Velusamy, R. (2021). Sagru: A stacked autoencoder-based gated recurrent unit approach to intrusion detection. Advances in Intelligent Systems and Computing, 1177. https://doi.org/10.1007/978-981-15-5679-1_5

Bhuyan, M. H., Bhattacharyya, D. K., & Kalita, J. K. (2014). Network anomaly detection: Methods, systems and tools. IEEE Communications Surveys and Tutorials, 16(1). https://doi.org/10.1109/SURV.2013.052213.00046

Chen, Z., Jiang, C., Masood, M. K., Soh, Y. C., Wu, M., & Li, X. (2020). Deep learning for building occupancy estimation using environmental sensors. In Studies in Computational Intelligence. https://doi.org/10.1007/978-3-030-31760-7_11

CyberEdge Group. (2021). 2021 Cyberthreat Defense Report. https://cyber-edge.com/cdr/

El-Amir, H., & Hamdy, M. (2020). Deep Learning Pipeline: Building a Deep Learning Model with TensorFlow. Apress.

Gamage, S., & Samarabandu, J. (2020). Deep learning methods in network intrusion detection: A survey and an objective comparison. Journal of Network and Computer Applications, 169. https://doi.org/10.1016/j.jnca.2020.102767

Gharib, A., Sharafaldin, I., Lashkari, A. H., & Ghorbani, A. A. (2017). An Evaluation Framework for Intrusion Detection Dataset. ICISS 2016 - 2016 International Conference on Information Science and Security. https://doi.org/10.1109/ICISSEC.2016.7885840

Goodfellow, I. J., Bengio, Y., & Courville, A. (2016). Deep Learning. MIT Press. www.deeplearningbook.org

Graves, A., & Schmidhuber, J. (2005). Framewise phoneme classification with bidirectional LSTM networks. Proceedings. 2005 IEEE International Joint Conference on Neural Networks, 2005., 4, 2047–2052 vol. 4. https://doi.org/10.1109/IJCNN.2005.1556215

Hochreiter, S., & Urgen Schmidhuber, J. J. (1997). Long Short-Term Memroy. In Neural Computation.

Imrana, Y., Xiang, Y., Ali, L., & Abdul-Rauf, Z. (2021). A bidirectional LSTM deep learning approach for intrusion detection. Expert Systems with Applications, 185, 115524. https://doi.org/https://doi.org/10.1016/j.eswa.2021.115524

K., V., & K., S. (2020). Towards activation function search for long short-term model network: A differential evolution based approach. Journal of King Saud University - Computer and Information Sciences. https://doi.org/https://doi.org/10.1016/j.jksuci.2020.04.015

Khan, M. A. (2021). HCRNNIDS: Hybrid convolutional recurrent neural network-based network intrusion detection system. Processes, 9(5). https://doi.org/10.3390/pr9050834

Kim, A., Park, M., & Lee, D. H. (2020). AI-IDS: Application of Deep Learning to Real-Time Web Intrusion Detection. IEEE Access, 8, 70245–70261. https://doi.org/10.1109/ACCESS.2020.2986882

Kingma, D. P., & Ba, J. L. (2015). Adam: A method for stochastic optimization. 3rd International Conference on Learning Representations, ICLR 2015 - Conference Track Proceedings.

Le, T.-T.-H., Kim, Y., & Kim, H. (2019). Network Intrusion Detection Based on Novel Feature Selection Model and Various Recurrent Neural Networks. Applied Sciences, 9(7). https://doi.org/10.3390/app9071392

Leevy, J. L., & Khoshgoftaar, T. M. (2020). A survey and analysis of intrusion detection models based on CSE-CIC-IDS2018 Big Data. Journal of Big Data, 7(1). https://doi.org/10.1186/s40537-020-00382-x

Liu, H., & Lang, B. (2019). Machine learning and deep learning methods for intrusion detection systems: A survey. In Applied Sciences (Switzerland). https://doi.org/10.3390/app9204396

Lohiya, R., & Thakkar, A. (2021). Intrusion Detection Using Deep Neural Network with AntiRectifier Layer. In S. M. Thampi, J. Lloret Mauri, X. Fernando, R. Boppana, S. Geetha, & A. Sikora (Eds.), Applied Soft Computing and Communication Networks (pp. 89–105). Springer Singapore.

Mazini, M., Shirazi, B., & Mahdavi, I. (2019). Anomaly network-based intrusion detection system using a reliable hybrid artificial bee colony and AdaBoost algorithms. Journal of King Saud University - Computer and Information Sciences, 31(4), 541–553. https://doi.org/https://doi.org/10.1016/j.jksuci.2018.03.011

Moolayil, J. (2019). Learn Keras for Deep Neural Networks: A Fast-Track Approach to Modern Deep Learning with Python. Apress. https://doi.org/https://doi.org/10.1007/978-1-4842-4240-7

Muhammad, A. U., Yahaya, A. S., Kamal, S. M., Adam, J. M., Muhammad, W. I., & Elsafi, A. (2020). A Hybrid Deep Stacked LSTM and GRU for Water Price Prediction. 2020 2nd International Conference on Computer and Information Sciences (ICCIS), 1–6. https://doi.org/10.1109/ICCIS49240.2020.9257651

Nayyar, S., Arora, S., & Singh, M. (2020). Recurrent Neural Network Based Intrusion Detection System. 2020 International Conference on Communication and Signal Processing (ICCSP), 136–140. https://doi.org/10.1109/ICCSP48568.2020.9182099

Ni, R., & Cao, H. (2020). Sentiment Analysis based on GloVe and LSTM-GRU. 2020 39th Chinese Control Conference (CCC), 7492–7497. https://doi.org/10.23919/CCC50068.2020.9188578

Ramasamy, M., & Eric, P. V. (2022). An improved deep bagging convolutional neural network classifier for efficient intrusion detection system. Bulletin of Electrical Engineering and Informatics, 11(1), 405–413.

Ripley, B. D. (2008). Pattern recognition and neural networks (1st ed.). Cambridge University Press.

Schuster, M., & Paliwal, K. K. (1997). Bidirectional recurrent neural networks. IEEE Transactions on Signal Processing. https://doi.org/10.1109/78.650093

Sharafaldin, I., Lashkari, A. H., & Ghorbani, A. A. (2018). Toward generating a new intrusion detection dataset and intrusion traffic characterization. ICISSP 2018 - Proceedings of the 4th International Conference on Information Systems Security and Privacy. https://doi.org/10.5220/0006639801080116

Thapa, N., Liu, Z., Kc, D. B., Gokaraju, B., & Roy, K. (2020). Comparison of machine learning and deep learning models for network intrusion detection systems. Future Internet. https://doi.org/10.3390/fi12100167

Wang, K., Huang, Y., Gong, L., Cai, C., & Zhang, Y. (2019). State-Wise LSTM-GRU Method for Ball Screw Prediction. 2019 IEEE Aerospace Conference, 1–8. https://doi.org/10.1109/AERO.2019.8741555

Xu, C., Shen, J., Du, X., & Zhang, F. (2018). An Intrusion Detection System Using a Deep Neural Network with Gated Recurrent Units. IEEE Access, 6. https://doi.org/10.1109/ACCESS.2018.2867564

Yan, W. (2021). Computational methods for deep learning. Springer.

Zhang, D., & Kabuka, M. R. (2018). Combining Weather Condition Data to Predict Traffic Flow: A GRU Based Deep Learning Approach. Proceedings - 2017 IEEE 15th International Conference on Dependable, Autonomic and Secure Computing, 2017 IEEE 15th International Conference on Pervasive Intelligence and Computing, 2017 IEEE 3rd International Conference on Big Data Intelligence and Compu. https://doi.org/10.1109/DASC-PICom-DataCom-CyberSciTec.2017.194

Zhu, Q., Cao, W., & Song, W. (2021). Multi-condition recognition method based on LSTM_GRU for heating processes. 2021 40th Chinese Control Conference (CCC), 6544–6549. https://doi.org/10.23919/CCC52363.2021.9550672




DOI: http://dx.doi.org/10.35671/telematika.v15i2.1855

Refbacks

  • There are currently no refbacks.


 


Indexed by:

   

Telematika
ISSN: 2442-4528 (online) | ISSN: 1979-925X (print)
Published by : Universitas Amikom Purwokerto
Jl. Let. Jend. POL SUMARTO Watumas, Purwonegoro - Purwokerto, Indonesia


Creative Commons License This work is licensed under a Creative Commons Attribution 4.0 International License .