FROM REACTIVE TO PREDICTIVE: HARNESSING KEDA AND CUSTOM MACHINE LEARNING FOR DYNAMIC POD MANAGEMENT
Keywords:
Kubernetes Autoscaling, KEDA, Machine Learning, Predictive Analytics, Dynamic Pod ManagementAbstract
This paper presents a framework for enhancing Kubernetes autoscaling by integrating KEDA with a custom machine-learning algorithm. Unlike traditional autoscaling methods that rely on static thresholds, this approach leverages predictive analytics to dynamically adjust the number of pods based on anticipated workloads. The proposed system aims to optimize resource utilization, reduce costs, and improve application performance through intelligent scaling decisions. We discuss the algorithm design, hypothetical scenarios, and the expected benefits and challenges of this approach.
References
Burns, B., Grant, B., Oppenheimer, D., Brewer, E., & Wilkes, J. (2016). Borg, Omega, and Kubernetes: Lessons learned from three container-management systems over a decade. ACM Queue, 14(1), 70-93.
Hindman, B., Konwinski, A., Zaharia, M., Ghodsi, A., Joseph, A. D., Katz, R. H., ... & Stoica, I. (2011). Mesos: A platform for fine-grained resource sharing in the data center. NSDI'11: Proceedings of the 8th USENIX Conference on Networked Systems Design and Implementation, 295-308.
Magalhães, S. A., Souza, F. F., & Santanna, F. S. (2020). KEDA: A Kubernetes-based event-driven autoscaler. 2020 IEEE International Conference on Cloud Engineering (IC2E), 230-236.
Horovitz, I., & Weiss, T. (2021). Kubernetes Autoscaling with Machine Learning. O'Reilly Media.
Klein, D., & Duarte, E. (2019). Machine Learning Approaches for Predictive Autoscaling in Containerized Cloud Applications. 2019 IEEE International Conference on Cloud Computing Technology and Science (CloudCom), 159-166.
Mao, M., & Humphrey, M. (2012). A performance study on the VM startup time in the cloud. 2012 IEEE Fifth International Conference on Cloud Computing, 423-430.
Villamizar, M., Garcés, O., Ochoa, L., Castro, H., Salamanca, L., Verano, M., ... & Lang, M. (2015). Infrastructure Cost Comparison of Running Web Applications in the Cloud Using AWS Lambda and AWS EC2. Proceedings of the 2015 IEEE/ACM 8th International Conference on Utility and Cloud Computing (UCC), 285-292.
Rashid, M., Ravichandran, M., & Gehani, A. (2019). Machine Learning-based Resource Scaling in Cloud Platforms: A Review, Trends, and Challenges. Journal of Cloud Computing: Advances, Systems and Applications, 8(1), 1-28.
Sharma, U., Shenoy, P., Sahu, S., & Shaikh, A. (2011). A cost-aware elasticity provisioning system for the cloud. 2011 31st International Conference on Distributed Computing Systems, 559-570.
Xavier, M. G., Neves, M. V., & Ururahy, C. G. (2013). Performance evaluation of container-based virtualization for high-performance computing environments. Proceedings of the 21st International Symposium on High-Performance Parallel and Distributed Computing, 91-102.
Moreno-Vozmediano, R., Montero, R. S., & Llorente, I. M. (2019). Elasticity management in cloud computing: State of the art and research challenges. Journal of Network and Computer Applications, 85, 38-54.
Vahid D., & Calheiros, R. N. (2015). A Survey of Techniques for Architecting Autoscaling of Web Applications on Cloud Computing. ACM Computing Surveys (CSUR), 47(4), 1-45.
Kumar, A., & Helal, A. (2014). Cloud autoscaling with machine learning. Proceedings of the 4th ACM International Conference on Pervasive Technologies Related to Assistive Environments, 1-8.
Zhu, X., Young, C., Watson, B. J., Rolia, J., Singhal, S., King, R., ... & Gmach, D. (2008). 1000 Islands: Integrated Capacity and Workload Management for the Next Generation Data Center. Proceedings of the 5th International Conference on Autonomic Computing (ICAC), 172-181.
Islam, S., Keung, J., Lee, K., & Liu, A. (2012). Empirical prediction models for adaptive resource provisioning in the cloud. Future Generation Computer Systems, 28(1), 155-162.
Shah, M. A., & Sarkar, S. (2017). A Survey of Predictive Modeling for Virtual Machine Auto-scaling. Journal of Cloud Computing: Advances, Systems and Applications, 6(1), 1-17.
Moldovan, D., & Copil, G. (2017). Elastic Load Prediction for Distributed Cloud Applications. Proceedings of the 2017 IEEE International Conference on Cloud Engineering (IC2E), 182-192.
Chen, Y., Alspaugh, S., & Katz, R. H. (2012). Interactive query processing in big data systems: A cross-industry study of MapReduce workloads. Proceedings of the VLDB Endowment, 5(12), 1802-1813.
Moreno, I., Garraghan, P., Townend, P., & Xu, J. (2014). Analysis, modeling, and simulation of workload patterns in a large-scale utility cloud. IEEE Transactions on Cloud Computing, 2(2), 208-221.
Carpio, F., & Bezerra, E. A. (2021). Using Machine Learning to Enhance Autoscaling in Kubernetes. Proceedings of the 2021 IEEE International Conference on Cloud Computing (CLOUD), 215-222.
Downloads
Published
Issue
Section
License
Copyright (c) 2024 Prashanth Lakshmi Narayana Chaitanya Josyula (Author)
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
