A Study on Metrics for Concept Drift Detection Based on Predictions and Parameters of Ensemble Model

The performance of machine learning models deteriorates when the distribution of test data changes, which is called concept drift. One way to deal with concept drift is to continuously rebuild the model. If we want to minimize the frequency of rebuilding due to some constraints, however, it is important to detect concept drift as the timing when rebuilding is truly necessary. Taking advantage of ensemble models for concept drift detection may improve the detection accuracy. However, the behavior of ensemble model’s predictions and parameters in the presence of concept drift has not been fully investigated. In this study, we investigated how the ensemble models constructed by two different methods behave in the presence of concept drift. In the experiments, we monitored some metrics including the metrics that can be calculated only by the ensemble model and the metrics based on the model parameters. As a result, we found that the metrics show some behaviors that seem to be influenced by concept drift, suggesting that the detection accuracy of concept drift may be improved by using these metrics.