Enhanced detection of denial-of-service attacks in Kubernetes: a multi-framework machine learning approach integrating node and application metrics

The widespread adoption of Kubernetes as a platform for orchestrating containerized applications has heightened the need for effective security mechanisms, particularly to counter Denial-of-Service (DoS) attacks. This article proposes an approach to DoS attack detection based on two key components the use of comprehensive metrics and the application of ensemble Machine Learning models. The approach involves the collection and analysis of comprehensive metrics from node-level (CPU, memory) and application-level (network activity, file descriptors) data from containers running on various frameworks (Flask, Django, FastAPI, Node.js, Golang). To implement this approach, a dataset containing 49,990 instances of network activity, characterized by 28 features (comprehensive metrics), was created. Statistical analysis (Student’s t-test, Pearson correlation) identified the metrics most relevant for attack detection, including total CPU time (cpu_sec_total) and resident memory usage (resident_memory_total). A comparison of nine Machine Learning models for attack detection was conducted, including ensemble methods (Random Forest, XGBoost, LightGBM) which demonstrated the highest effectiveness, achieving 100 % accuracy (F1-score equals 1.0) and perfect class separation (AUC equals 1.0). The XGBoost model also eliminated false positives (precision equals 1.0). Feature importance analysis revealed the most significant metrics for classification: CPU usage (cpu_sec_total, cpu_sec_idle), network packet transmission (transmit_packets), system load average, and memory usage (virtual_memory_total, resident_ memory_total). The work emphasizes the importance of integrating multi-level metrics for building resilient anomaly detection systems. The proposed approach is scalable and independent of specific frameworks, making it applicable for protecting containerized environments. The research results serve as a foundation for developing proactive Kubernetes security systems capable of countering sophisticated attack vectors.

1. Sadiq A., Syed H.J., Ansari A.A., Ibrahim A.O., Alohaly M., Elsadig M. Detection of denial of service attack in cloud based kubernetes using eBPF // Applied Sciences. 2023. V. 13. N 8. P. 4700. https://doi.org/10.3390/app13084700

2. Cao C., Blaise A., Verwer S., Rebecchi F. Learning state machines to monitor and detect anomalies on a kubernetes cluster // Proc. of the 17th International Conference on Availability, Reliability and Security. 2022. P. 1–9. https://doi.org/10.1145/3538969.3543810

3. Koksal S., Catak F. O., Dalveren Y. Flexible and lightweight mitigation framework for distributed denial-of-service attacks in container-based edge networks using Kubernetes // IEEE Access. 2024. V. 12. P. 172980–172991. https://doi.org/10.1109/ACCESS.2024.3501192

4. Tripathi A.A. Attacking and Defending Kubernetes. PhD thesis. Dublin Business School, 2024. [Online]. URL: https://esource.dbs.ie/items/eda4ea15-cedf-456b-93f9-6ce67e25c4bb (accessed: 02.12.2024).

5. Darwesh G., Hammoud J., Vorobeva A.A. Enhancing Kubernetes security with machine learning: а proactive approach to anomaly detection // Scientific and Technical Journal of Information Technologies, Mechanics and Optics. 2024. V. 24. N 6. P. 1007–1015. https://doi.org/10.17586/2226-1494-2024-24-6-1007-1015

6. Ghadeer D., Jaafar H., Vorobeva A.A. Security in Kubernetes: best practices and security analysis // Journal of the Ural Federal District. Information Security. 2022. N. 2 (44). P. 63–69. https://doi.org/10.14529/secur220209

7. Darwesh G., Hammoud J., Vorobeva A.A. Enhancing kubernetes security: the crucial role of DevSecOps // Proc. of the Institute for Systems Analysis Russian Academy of Sciences. 2024. V. 74. N 3. P. 78-88. https://doi.org/10.14357/20790279240309

8. Abed A.S., Clancy C., Levy D.S. Intrusion detection system for applications using linux containers // Lecture Notes in Computer Science. 2024. V. 9331. P. 123–135. https://doi.org/10.1007/978-3-319-24858-5_8

9. Zou Z., Xie Y., Huang K., Xu G., Feng D., Long D. A docker container anomaly monitoring system based on optimized isolation forest // IEEE Transactions on Cloud Computing. 2022. V. 10. N 1. P. 134–145. https://doi.org/10.1109/TCC.2019.2935724

10. Srinivasan S., Kumar A., Mahajan M., Sitaram D., Gupta S. Probabilistic real-time intrusion detection system for docker containers // Communications in Computer and Information Science. 2019. V. 969. P. 336–347. https://doi.org/10.1007/978-981-13-5826-5_26

11. Tunde-Onadele O., He J., Dai T., Gu X. A study on container vulnerability exploit detection // Proc. of the IEEE International Conference on Cloud Engineering (IC2E). 2019. P. 121–127. https://doi.org/10.1109/IC2E.2019.00026

12. Flora J., Gonçalves P., Antunes N. Using attack injection to evaluate intrusion detection effectiveness in container-based systems // Proc. of the IEEE 25th Pacific Rim International Symposium on Dependable Computing (PRDC). 2020. P. 60–69. https://doi.org/10.1109/PRDC50213.2020.00017

13. Haq M.S., Nguyen T.D., Tosun A.S., Vollmer F., Korkmaz T., Sadeghi A.-R. SoK: a comprehensive analysis and evaluation of docker container attack and defense mechanisms // Proc. of the IEEE Symposium on Security and Privacy (SP). 2024. P. 4573–4590. https://doi.org/10.1109/sp54263.2024.00268

14. Lin Y., Tunde-Onadele O., Gu X. Cdl: Classified distributed learning for detecting security attacks in containerized applications // Proc. of the 36th Annual Computer Security Applications Conference. 2020. P. 179–188. https://doi.org/10.1145/3427228.3427236

15. Darwesh G., Hammoud J., Vorobeva A.A. A novel approach to feature collection for anomaly detection in Kubernetes environment and agent for metrics collection from Kubernetes nodes // Scientific and Technical Journal of Information Technologies, Mechanics and Optics. 2023. V. 23. N 3. P. 538–546. https://doi.org/10.17586/2226-1494-2023-23-3-538-546

16. LaValley M.P. Logistic regression // Circulation. 2008. V. 117. N 18. P. 2395–2399. https://doi.org/10.1161/circulationaha.106.682658

17. Rigatti S.J. Random Forest // Journal of Insurance Medicine. 2017. V. 47. N 1. P. 31–39. https://doi.org/10.17849/insm-47-01-31-39.1

18. Natekin A., Knoll A. Gradient boosting machines, a tutorial // Frontiers in Neurorobotics. 2013. V. 7. P. 21. https://doi.org/10.3389/fnbot.2013.00021

19. Suthaharan S. Support vector machine // Integrated Series in Information Systems. 2016. V. 36. P. 207–235. https://doi.org/10.1007/978-1-4899-7641-3_9

20. Song Y., Lu Y. Decision tree methods: applications for classification and prediction // Shanghai Archives of Psychiatry. 2015. V. 27. N 2. P. 130–135, https://doi.org/10.11919/j.issn.1002-0829.215044

21. Rish I. An empirical study of the naive Bayes classifier // Proc. of the IJCAI-2001 Workshop on Empirical Methods in Artificial Intelligence. 2001. P. 41–46.

22. Kramer O. K-Nearest neighbors // Intelligent Systems Reference Library. 2013. V. 51. P. 13–23. https://doi.org/10.1007/978-3-642-38652-7_2

23. Chen T., Guestrin C. XGBoost: A Scalable Tree Boosting System // Proc. of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. 2016. P. 785–794. https://doi.org/10.1145/2939672.2939785

24. Ke G., Meng Q., Finley T., Wang T., Chen W., Ma W., Ye Q., Liu T.-Y. LightGBM: a highly efficient gradient boosting decision tree // Proc. of the 31st International Conference on Neural Information Processing Systems. 2017. P. 3149–3157.