Research on the Fusion Model of DeepFM and XGBoost for Digital Consumer Behavior Prediction

Haotian Zhou

doi:10.71451/ISTAER2617

Authors

Haotian Zhou School of Business Administration/School of Marxism, China University of Petroleum-Beijing at Karamay, Karamay, Xinjiang, China Author https://orcid.org/0009-0006-4551-2482

DOI:

https://doi.org/10.71451/ISTAER2617

Keywords:

DeepFM; XGBoost; Fusion model; Consumer behavior prediction; Dynamic weighted fusion

Abstract

To leverage the complementary characteristics of deep models and tree models in feature interaction modeling for digital consumer behavior prediction, this paper proposes a dual channel fusion model of DeepFM and XGBoost. In this model, an optimized DeepFM branch and an enhanced XGBoost branch are constructed using a feature shunting mechanism, and the dynamic weighted fusion and attention mechanism based on sample features are introduced to realize the adaptive combination of the two branch outputs. At the same time, a feature interaction enhancement algorithm is designed, which combines the depth implicit representation with the rule features of the tree model by multiplication, and further improves the depiction ability of high-order interaction. Experiments on real e-commerce user behavior and ad click through rate data sets show that the AUC of this model reaches 0.879, LogLoss drops to 0.342, which is 5.4% higher and 11.6% lower than DeepFM, and 8.3% higher and 14.7% lower than XGBoost, respectively. Ablation experiments verify the effectiveness of the dynamic weighted fusion and feature enhancement module, and the performance degradation is 6.7% and 10.5%, respectively. The robustness test showed that the AUC remained at 0.839 and 0.851 under the proportion of 30% missing features and 1% positive samples, and the click-through rate in online simulations increased to 4.93%, which was 1.6% higher than that of the industrial reference system. The proposed model has significant advantages in prediction accuracy and stability.

References

[1] Lin, J. (2025). Application of machine learning in predicting consumer behavior and precision marketing. PLoS One, 20(5), e0321854. DOI: https://doi.org/10.1371/journal.pone.0321854

[2] Theodorakopoulos, L., & Theodoropoulou, A. (2024). Leveraging big data analytics for understanding consumer behavior in digital marketing: A systematic review. Human Behavior and Emerging Technologies, 2024(1), 3641502. DOI: https://doi.org/10.1155/2024/3641502

[3] Yin, J., Qiu, X., & Wang, Y. (2025). The impact of AI-personalized recommendations on clicking intentions: Evidence from Chinese e-commerce. Journal of Theoretical and Applied Electronic Commerce Research, 20(1), 21. DOI: https://doi.org/10.3390/jtaer20010021

[4] Koosha, H., & Albadvi, A. (2020). Allocation of marketing budgets to maximize customer equity. Operational Research, 20(2), 561-583. DOI: https://doi.org/10.1007/s12351-017-0356-z

[5] Yan, C., Chen, Y., Wan, Y., & Wang, P. (2021). Modeling low-and high-order feature interactions with FM and self-attention network. Applied Intelligence, 51(6), 3189-3201. DOI: https://doi.org/10.1007/s10489-020-01951-6

[6] Chen, T., Yin, H., Zhang, X., Huang, Z., Wang, Y., & Wang, M. (2021). Quaternion factorization machines: A lightweight solution to intricate feature interaction modeling. IEEE Transactions on Neural Networks and Learning Systems, 34(8), 4345-4358. DOI: https://doi.org/10.1109/TNNLS.2021.3118706

[7] Zhu, Z. (2022, July). Deep Learning for FM-Based Recommendation: A Systematic Study on DeepFm and Its Application. In International Conference on Frontier Computing (pp. 747-756). Singapore: Springer Nature Singapore. DOI: https://doi.org/10.1007/978-981-99-1428-9_92

[8] Zhang, P., Tang, K., Chen, G., Li, J., & Li, Y. (2024). Multimodal data fusion enhanced deep learning prediction of crack path segmentation in CFRP composites. Composites Science and Technology, 257, 110812. DOI: https://doi.org/10.1016/j.compscitech.2024.110812

[9] Wang, K., Wang, H., Guo, W., Liu, Y., Lin, J., Lian, D., & Chen, E. (2025, July). DLF: Enhancing explicit-implicit interaction via dynamic low-order-aware fusion for CTR prediction. In Proceedings of the 48th International ACM SIGIR Conference on Research and Development in Information Retrieval (pp. 2213-2223). DOI: https://doi.org/10.1145/3726302.3729956

[10] Xu, K., Wang, T., & Cheng, L. (2023). Service recommendation of industrial software components based on explicit and implicit higher-order feature interactions and attentional factorization machines. Applied Sciences, 13(19), 10746. DOI: https://doi.org/10.3390/app131910746

[11] Liu, B., Zhu, C., Li, G., Zhang, W., Lai, J., Tang, R., ... & Yu, Y. (2020, August). Autofis: Automatic feature interaction selection in factorization models for click-through rate prediction. In proceedings of the 26th ACM SIGKDD international conference on knowledge discovery & data mining (pp. 2636-2645). DOI: https://doi.org/10.1145/3394486.3403314

[12] Cheng, W., Shen, Y., & Huang, L. (2020, April). Adaptive factorization network: Learning adaptive-order feature interactions. In Proceedings of the AAAI Conference on Artificial Intelligence (Vol. 34, No. 04, pp. 3609-3616). DOI: https://doi.org/10.1609/aaai.v34i04.5768

[13] Yu, Z., Amin, S. U., Alhussein, M., & Lv, Z. (2021). Research on disease prediction based on improved DeepFM and IoMT. IEEE Access, 9, 39043-39054. DOI: https://doi.org/10.1109/access.2021.3062687

[14] Sun, Y., Li, J., Xu, Y., Zhang, T., & Wang, X. (2023). Deep learning versus conventional methods for missing data imputation: A review and comparative study. Expert Systems with Applications, 227, 120201. DOI: https://doi.org/10.1016/j.eswa.2023.120201

[15] Emmanuel, T., Maupong, T., Mpoeleng, D., Semong, T., Mphago, B., & Tabona, O. (2021). A survey on missing data in machine learning. Journal of Big data, 8(1), 140. DOI: https://doi.org/10.21203/rs.3.rs-535520/v1

[16] Wang, H., Tan, Z., Liang, Y., Li, F., Zhang, Z., & Ju, L. (2024). A novel multi-layer stacking ensemble wind power prediction model under Tensorflow deep learning framework considering feature enhancement and data hierarchy processing. Energy, 286, 129409. DOI: https://doi.org/10.1016/j.energy.2023.129409

[17] Yu, F., & Liu, X. (2022). Research on student performance prediction based on stacking fusion model. Electronics, 11(19), 3166. DOI: https://doi.org/10.3390/electronics11193166

[18] Chen, M., Qian, Z., Boers, N., Jakeman, A. J., Kettner, A. J., Brandt, M., ... & Lü, G. (2023). Iterative integration of deep learning in hybrid Earth surface system modelling. Nature Reviews Earth & Environment, 4(8), 568-581. DOI: https://doi.org/10.1038/s43017-023-00452-7

[19] Ma, L., Yao, W., Dai, X., & Jia, R. (2023). A new evidence weight combination and probability allocation method in multi-sensor data fusion. Sensors, 23(2), 722. DOI: https://doi.org/10.3390/s23020722

[20] Huang, H., Yan, X., Zheng, Y., He, J., Xu, L., & Qin, D. (2025). Multi-view stereo algorithms based on deep learning: a survey. Multimedia Tools and Applications, 84(6), 2877-2908. DOI: https://doi.org/10.1007/s11042-024-20464-9

[21] Xie, Z., Yang, Y., Zhang, Y., Wang, J., & Du, S. (2023). Deep learning on multi-view sequential data: a survey. Artificial Intelligence Review, 56(7), 6661-6704. DOI: https://doi.org/10.1007/s10462-022-10332-z

[22] Dong, Y., Qiu, L., Lu, C., Song, L., Ding, Z., Yu, Y., & Chen, G. (2022). A data-driven model for predicting initial productivity of offshore directional well based on the physical constrained eXtreme gradient boosting (XGBoost) trees. Journal of Petroleum Science and Engineering, 211, 110176. DOI: https://doi.org/10.1016/j.petrol.2022.110176

[23] Demir, S., & Sahin, E. K. (2023). An investigation of feature selection methods for soil liquefaction prediction based on tree-based ensemble algorithms using AdaBoost, gradient boosting, and XGBoost. Neural Computing and Applications, 35(4), 3173-3190. DOI: https://doi.org/10.1007/s00521-022-07856-4

[24] Liu, Z., Lu, Y., Lai, Z., Ou, W., & Zhang, K. (2021). Robust sparse low-rank embedding for image dimension reduction. Applied soft computing, 113, 107907. DOI: https://doi.org/10.1016/j.asoc.2021.107907

[25] Guo, Y., Sun, Y., Wang, Z., Nie, F., & Wang, F. (2023). Double-structured sparsity guided flexible embedding learning for unsupervised feature selection. IEEE Transactions on Neural Networks and Learning Systems, 35(10), 13354-13367. DOI: https://doi.org/10.1109/tnnls.2023.3267184

[26] Singh, D., & Singh, B. (2022). Feature wise normalization: An effective way of normalizing data. Pattern Recognition, 122, 108307. DOI: https://doi.org/10.1016/j.patcog.2021.108307

[27] Sarmadi, H., Entezami, A., & Magalhães, F. (2023). Unsupervised data normalization for continuous dynamic monitoring by an innovative hybrid feature weighting-selection algorithm and natural nearest neighbor searching. Structural health monitoring, 22(6), 4005-4026. DOI: https://doi.org/10.1177/14759217231166116

[28] Huang, L., Qin, J., Zhou, Y., Zhu, F., Liu, L., & Shao, L. (2023). Normalization techniques in training dnns: Methodology, analysis and application. IEEE transactions on pattern analysis and machine intelligence, 45(8), 10173-10196. DOI: https://doi.org/10.1109/tpami.2023.3250241

[29] Zhu, Z. (2022, July). Deep Learning for FM-Based Recommendation: A Systematic Study on DeepFm and Its Application. In International Conference on Frontier Computing (pp. 747-756). Singapore: Springer Nature Singapore. DOI: https://doi.org/10.1007/978-981-99-1428-9_92

[30] Qian, J., Jia, T., Zhang, W., Zeng, K., & Du, X. (2024). An industrial network traffic anomaly detection method based on improved DeepFM model. IEEE Access, 12, 136222-136229. DOI: https://doi.org/10.1109/access.2024.3419895

[31] Yan, C., Chen, Y., Wan, Y., & Wang, P. (2021). Modeling low-and high-order feature interactions with FM and self-attention network. Applied Intelligence, 51(6), 3189-3201. DOI: https://doi.org/10.1007/s10489-020-01951-6

[32] Zhai, Z., Shen, J., Li, P., Zhang, J., & Zhang, K. (2024, December). Flexible-Order Feature-Interaction for Mixed Continuous and Discrete Variables with Group-Level Interpretability. In International Conference on Neural Information Processing (pp. 42-57). Singapore: Springer Nature Singapore. DOI: https://doi.org/10.1007/978-981-96-6576-1_4