Machine Learning for Enhanced Churn Prediction in Banking: Leveraging Oversampling and Stacking Techniques

Omar Faruq; Fahad Ahammed; Arifa Sultana Mily; 4Ashraful Islam

doi:10.18535/ijsrm/v12i09.ec03

Abstract

Every sector of business is getting more competitive as time passes. More and more companies are offering services to people. Banking sector is no different. With the plethora option customer has in terms banking, holding on to customer may prove difficult for banks. This research will help banks to predict which customers are likely going to churn and allow them to take precaution to stop customers from leaving. In this study we have used classifiers such as: K-Neighbors, Random Forest, XGboost, Adaboost classifiers and Ensemble Model (Stacking Technique) that uses all of these models together. The experimentation was conducted on a dataset from Kaggle. The dataset used in this research was heavily imbalanced. So, different oversampling methods like Random Oversampling and SMOTE-ENN have been used. In data preprocessing, label encoding was done and for validation K-folding technique (k-5) have been used. The highest accuracy has been achieved by using Random Oversampling with Stacking Model which is 97.31% (std: 0.0033, k=5).

Keywords

Customer churn PredictionMachine LearningOversamplingEnsemble ModelK-fold

References

S Neslin, S Gupta, W Kamakura, J Lu, C Mason,"Defection Detection: Improving Predictive Accuracy of Customer Chum Models", Working Paper, Teradata Center at Duke University, 2004.3Google Scholar ↗
Zorić, A. B. (2016). Predicting Customer Churn in Banking Industry using Neural Networks. Interdisciplinary Description of Complex Systems, 14(2), 116–124. https://doi.org/10.7906/indecs.14.2.1DOI ↗Google Scholar ↗
A Lemmens, C Croux, "Bagging and Boosting Classification Trees to predict churn", Journal of Marketing Research, Vol. 43, No. 2, pp 276286.2006Google Scholar ↗
M.-K. Kim, M.-C. Park, and D.-H. Jeong, “The effects of customer satisfaction and switching barrier on customer loyalty in Korean mobile telecommunication services,” Telecommunications policy, vol. 28, no. 2, pp. 145–159, 2004Google Scholar ↗
Y. Deng, D. Li, L. Yang, J. Tang and J. Zhao, "Analysis and prediction of bank user churn based on ensemble learning algorithm," 2021 IEEE International Conference on Power Electronics, Computer Applications (ICPECA), Shenyang, China, 2021, pp. 288-291, doi: 10.1109/ICPECA51329.2021.9362520.DOI ↗Google Scholar ↗
Verma, Prashant (2020) "Churn Prediction for Savings Bank Customers: A Machine Learning Approach," Journal of Statistics Applications & Probability: Vol. 9: Iss. 3, Article 10.Google Scholar ↗
S. Jinbo, Li Xiu and L. Wenhuang, "The Application ofAdaBoost in Customer Churn Prediction," 2007 International Conference on Service Systems and Service Management, Chengdu, China, 2007, pp. 1-6, doi:Google Scholar ↗
10.1109/ICSSSM.2007.4280172.M. Young, The Technical Writer’s Handbook. Mill Valley, CA: University Science, 1989.DOI ↗Google Scholar ↗
M. Rahman and V. Kumar, "Machine Learning Based Customer Churn Prediction In Banking," 2020 4th International Conference on Electronics, Communication and Aerospace Technology (ICECA), Coimbatore, India, 2020, pp. 1196-1201, doi: 10.1109/ICECA49313.2020.9297529DOI ↗Google Scholar ↗
S. Cui and N. Ding, "Customer churn prediction using improved FCM algorithm," 2017 3rd International Conference on Information Management (ICIM), Chengdu, China, 2017, pp. 112-117, doi: 10.1109/INFOMAN.2017.7950357.DOI ↗Google Scholar ↗
Bharathi S V, Pramod D, Raman R. An Ensemble Model for Predicting Retail Banking Churn in the Youth Segment of Customers. Data. 2022; 7(5):61. https://doi.org/10.3390/data7050061M. Young, The Technical Writer’s Handbook. Mill Valley, CA: University Science, 1989.DOI ↗Google Scholar ↗
Muneer, A., Ali, R. F., Alghamdi, A., Taib, S. M., Almaghthawi, A., & Ghaleb, E. a. A. (2022). Predicting customers churning in banking industry: A machine learning approach. Indonesian Journal of Electrical Engineering and Computer Science, 26(1), 539. https://doi.org/10.11591/ijeecs.v26.i1.pp539-549M. Young, The Technical Writer’s Handbook. Mill Valley, CA: University Science, 1989.DOI ↗Google Scholar ↗
Iranmanesh, Seyed Hossein, et al. "Customer Churn Prediction Using Artificial Neural Network: An Analytical CRM Application." 3rd European International Conference on Industrial Engineering and Operations Management. 2019,M. Young, The Technical Writer’s Handbook. Mill Valley, CA: University Science, 1989.Google Scholar ↗
Xie, Y., Li, X., Ngai, E. W., & Ying, W. (2009). Customer churn prediction using improved balanced random forests. Expert SystemsGoogle Scholar ↗
With Applications, 36(3), 5445–5449.Google Scholar ↗
https://doi.org/10.1016/j.eswa.2008.06.121DOI ↗Google Scholar ↗
Dierckx, G. (2004). Logistic Regression model. Encyclopedia of Actuarial Science. https://doi.org/10.1002/9780470012505.tal017M. Young, The Technical Writer’s Handbook. Mill Valley, CA: University Science, 1989.DOI ↗Google Scholar ↗
Slimani, C., Wu, C., Rubini, S., Chang, Y., & Boukhobza, J. (2023). Accelerating random forest on Memory-Constrained Devices through data storage optimization. IEEE Transactions on Computers, 72(6), 1595–1609. https://doi.org/10.1109/tc.2022.3215898M. Young, The Technical Writer’s Handbook. Mill Valley, CA: University Science, 1989.DOI ↗Google Scholar ↗
Kutschenreiter-Praszkiewicz, I. (2023). Development of a neural network structure for identifying begin-end points in the assembly process. Journal of Machine Engineering. https://doi.org/10.36897/jme/163318M. Young, The Technical Writer’s Handbook. Mill Valley, CA: University Science, 1989.DOI ↗Google Scholar ↗
Medvedev, A., Delvenne, J., & Lambiotte, R. (2018). Modelling structure and predicting dynamics of discussion threads in online boards. Journal of Complex Networks, 7(1), 67–82. https://doi.org/10.1093/comnet/cny010M. Young, The Technical Writer’s Handbook. Mill Valley, CA: University Science, 1989.DOI ↗Google Scholar ↗
Zhang, Y., & Wang, L. (2023). An AdaBoost Method with K′KMeans Bayes Classifier for Imbalanced Data. Mathematics, 11(8), 1878. https://doi.org/10.3390/math11081878.DOI ↗Google Scholar ↗
Singh, P. (2023). Intepretable Deep Gaussian Naive Bayes Algorithm (IDGNBA) based task offloading framework for Edge-Cloud computing. International Journal of Data Informatics and Intelligent Computing, 2(2), 1–10. https://doi.org/10.59461/ijdiic.v2i2.57DOI ↗Google Scholar ↗
Breiman, L. (2001). Random forests. Machine Learning, 45(1), 5-32. https://doi.org/10.1023/A:1010933404324DOI ↗Google Scholar ↗
Freund, Y., & Schapire, R. E. (1997). A decision-theoretic generalization of on-line learning and an application to boosting. Journal of Computer and System Sciences, 55(1), 119-139. https://doi.org/10.1006/jcss.1997.1504DOI ↗Google Scholar ↗
Cover, T., & Hart, P. (1967). Nearest-neighbor pattern classification. IEEE Transactions on Information Theory, 13(1), 21-27. https://doi.org/10.1109/TIT.1967.1053964DOI ↗Google Scholar ↗
Chen, T., & Guestrin, C. (2016). XGBoost: A scalable tree boosting system. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (pp. 785-794). https://doi.org/10.1145/2939672.2939785DOI ↗Google Scholar ↗
Wolpert, D. H. (1992). Stacked generalization. Neural Networks, 5(2), 241-259. https://doi.org/10.1016/S0893-6080(05)80023-1DOI ↗Google Scholar ↗
Kohavi, R. (1995). A study of cross-validation and bootstrap for accuracy estimation and model selection. In Proceedings of the 14th International Joint Conference on Artificial Intelligence (pp. 1137-1143). https://dl.acm.org/doi/10.5555/1643031.1643047DOI ↗Google Scholar ↗

[refR-1] S Neslin, S Gupta, W Kamakura, J Lu, C Mason,"Defection Detection: Improving Predictive Accuracy of Customer Chum Models", Working Paper, Teradata Center at Duke University, 2004.3Google Scholar ↗

[refR-2] Zorić, A. B. (2016). Predicting Customer Churn in Banking Industry using Neural Networks. Interdisciplinary Description of Complex Systems, 14(2), 116–124. https://doi.org/10.7906/indecs.14.2.1DOI ↗Google Scholar ↗

[refR-3] A Lemmens, C Croux, "Bagging and Boosting Classification Trees to predict churn", Journal of Marketing Research, Vol. 43, No. 2, pp 276286.2006Google Scholar ↗

[refR-4] M.-K. Kim, M.-C. Park, and D.-H. Jeong, “The effects of customer satisfaction and switching barrier on customer loyalty in Korean mobile telecommunication services,” Telecommunications policy, vol. 28, no. 2, pp. 145–159, 2004Google Scholar ↗

[refR-5] Y. Deng, D. Li, L. Yang, J. Tang and J. Zhao, "Analysis and prediction of bank user churn based on ensemble learning algorithm," 2021 IEEE International Conference on Power Electronics, Computer Applications (ICPECA), Shenyang, China, 2021, pp. 288-291, doi: 10.1109/ICPECA51329.2021.9362520.DOI ↗Google Scholar ↗

[refR-6] Verma, Prashant (2020) "Churn Prediction for Savings Bank Customers: A Machine Learning Approach," Journal of Statistics Applications & Probability: Vol. 9: Iss. 3, Article 10.Google Scholar ↗

[refR-7] S. Jinbo, Li Xiu and L. Wenhuang, "The Application ofAdaBoost in Customer Churn Prediction," 2007 International Conference on Service Systems and Service Management, Chengdu, China, 2007, pp. 1-6, doi:Google Scholar ↗

[refR-8] 10.1109/ICSSSM.2007.4280172.M. Young, The Technical Writer’s Handbook. Mill Valley, CA: University Science, 1989.DOI ↗Google Scholar ↗

[refR-9] M. Rahman and V. Kumar, "Machine Learning Based Customer Churn Prediction In Banking," 2020 4th International Conference on Electronics, Communication and Aerospace Technology (ICECA), Coimbatore, India, 2020, pp. 1196-1201, doi: 10.1109/ICECA49313.2020.9297529DOI ↗Google Scholar ↗

[refR-10] S. Cui and N. Ding, "Customer churn prediction using improved FCM algorithm," 2017 3rd International Conference on Information Management (ICIM), Chengdu, China, 2017, pp. 112-117, doi: 10.1109/INFOMAN.2017.7950357.DOI ↗Google Scholar ↗

[refR-11] Bharathi S V, Pramod D, Raman R. An Ensemble Model for Predicting Retail Banking Churn in the Youth Segment of Customers. Data. 2022; 7(5):61. https://doi.org/10.3390/data7050061M. Young, The Technical Writer’s Handbook. Mill Valley, CA: University Science, 1989.DOI ↗Google Scholar ↗

[refR-12] Muneer, A., Ali, R. F., Alghamdi, A., Taib, S. M., Almaghthawi, A., & Ghaleb, E. a. A. (2022). Predicting customers churning in banking industry: A machine learning approach. Indonesian Journal of Electrical Engineering and Computer Science, 26(1), 539. https://doi.org/10.11591/ijeecs.v26.i1.pp539-549M. Young, The Technical Writer’s Handbook. Mill Valley, CA: University Science, 1989.DOI ↗Google Scholar ↗

[refR-13] Iranmanesh, Seyed Hossein, et al. "Customer Churn Prediction Using Artificial Neural Network: An Analytical CRM Application." 3rd European International Conference on Industrial Engineering and Operations Management. 2019,M. Young, The Technical Writer’s Handbook. Mill Valley, CA: University Science, 1989.Google Scholar ↗

[refR-14] Xie, Y., Li, X., Ngai, E. W., & Ying, W. (2009). Customer churn prediction using improved balanced random forests. Expert SystemsGoogle Scholar ↗

[refR-15] With Applications, 36(3), 5445–5449.Google Scholar ↗

[refR-16] https://doi.org/10.1016/j.eswa.2008.06.121DOI ↗Google Scholar ↗

[refR-17] Dierckx, G. (2004). Logistic Regression model. Encyclopedia of Actuarial Science. https://doi.org/10.1002/9780470012505.tal017M. Young, The Technical Writer’s Handbook. Mill Valley, CA: University Science, 1989.DOI ↗Google Scholar ↗

[refR-18] Slimani, C., Wu, C., Rubini, S., Chang, Y., & Boukhobza, J. (2023). Accelerating random forest on Memory-Constrained Devices through data storage optimization. IEEE Transactions on Computers, 72(6), 1595–1609. https://doi.org/10.1109/tc.2022.3215898M. Young, The Technical Writer’s Handbook. Mill Valley, CA: University Science, 1989.DOI ↗Google Scholar ↗

[refR-19] Kutschenreiter-Praszkiewicz, I. (2023). Development of a neural network structure for identifying begin-end points in the assembly process. Journal of Machine Engineering. https://doi.org/10.36897/jme/163318M. Young, The Technical Writer’s Handbook. Mill Valley, CA: University Science, 1989.DOI ↗Google Scholar ↗

[refR-20] Medvedev, A., Delvenne, J., & Lambiotte, R. (2018). Modelling structure and predicting dynamics of discussion threads in online boards. Journal of Complex Networks, 7(1), 67–82. https://doi.org/10.1093/comnet/cny010M. Young, The Technical Writer’s Handbook. Mill Valley, CA: University Science, 1989.DOI ↗Google Scholar ↗

[refR-21] Zhang, Y., & Wang, L. (2023). An AdaBoost Method with K′KMeans Bayes Classifier for Imbalanced Data. Mathematics, 11(8), 1878. https://doi.org/10.3390/math11081878.DOI ↗Google Scholar ↗

[refR-22] Singh, P. (2023). Intepretable Deep Gaussian Naive Bayes Algorithm (IDGNBA) based task offloading framework for Edge-Cloud computing. International Journal of Data Informatics and Intelligent Computing, 2(2), 1–10. https://doi.org/10.59461/ijdiic.v2i2.57DOI ↗Google Scholar ↗

[refR-23] Breiman, L. (2001). Random forests. Machine Learning, 45(1), 5-32. https://doi.org/10.1023/A:1010933404324DOI ↗Google Scholar ↗

[refR-24] Freund, Y., & Schapire, R. E. (1997). A decision-theoretic generalization of on-line learning and an application to boosting. Journal of Computer and System Sciences, 55(1), 119-139. https://doi.org/10.1006/jcss.1997.1504DOI ↗Google Scholar ↗

[refR-25] Cover, T., & Hart, P. (1967). Nearest-neighbor pattern classification. IEEE Transactions on Information Theory, 13(1), 21-27. https://doi.org/10.1109/TIT.1967.1053964DOI ↗Google Scholar ↗

[refR-26] Chen, T., & Guestrin, C. (2016). XGBoost: A scalable tree boosting system. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (pp. 785-794). https://doi.org/10.1145/2939672.2939785DOI ↗Google Scholar ↗

[refR-27] Wolpert, D. H. (1992). Stacked generalization. Neural Networks, 5(2), 241-259. https://doi.org/10.1016/S0893-6080(05)80023-1DOI ↗Google Scholar ↗

[refR-28] Kohavi, R. (1995). A study of cross-validation and bootstrap for accuracy estimation and model selection. In Proceedings of the 14th International Joint Conference on Artificial Intelligence (pp. 1137-1143). https://dl.acm.org/doi/10.5555/1643031.1643047DOI ↗Google Scholar ↗