The Improved XdeepFM Algorithm Based on Attention Mechanism and Factorization Machine

This study proposes an improved recommendation model called MHSA-XdeepFM, which incorporates multi-head self-attention mechanism and an enhanced residual network into XdeepFM to enhance the performance of click-through rate prediction tasks. Click-through rate prediction is one of the key tasks in recommendation systems, aiming to predict the probability of users clicking on candidate items. XDeepFM is a click-through rate prediction model that combines deep neural networks(DNN) with Compressed Interaction Network (CIN) to capture both low-order and high-order feature interactions, while introducing linear layers to emphasize the importance of first-order features. However, in its final output, XDeepFM simply concatenates the outputs of various sub-models without fully considering the connections between them and the importance of features, which may lead to information redundancy and imbalance in feature fusion and representation, resulting in poor accuracy in click-through rate prediction. To address this issue, the study introduces a multi-head self-attention mechanism that allows the model to adaptively focus on the importance of different sub-models. Additionally, the Adaptive Feature Interaction Modeling AFM is incorporated to adaptively model second-order feature interactions. Through these improvements, the model can better capture the correlations and interaction patterns between features, thus improving the accuracy and effectiveness of click-through rate prediction. Experimental results demonstrate that the model outperforms the traditional XdeepFM on the publicly available Criteo dataset, providing important improvements and guidance for the application of recommendation systems in real-world scenarios. The model shows enhancements in performance metrics such as AUC and LogLoss, proving the practical significance of this research in improving the modeling of feature interactions in click-through rate prediction tasks.