Anomaly Detection of Unstable Log Data Based on Contrastive Learning

In today’s large computer systems, it is almost impossible to locate errors using traditional manual methods in the face of abnormal conditions due to the unprecedented size and complexity of the system. As part of computer resources, log data records every step of the system operation process, making log data a popular method for detecting abnormal system status. However, in actual production environments, normal logs make up most of the log data types, and the log structure is constantly adjusted with each update for maintenance and upgrades. Therefore, overcoming the instability of logs caused by these issues has become a major concern for researchers in the field of log detection. In this paper, we propose a method to improve log instability and enhance model detection accuracy by using contrastive learning in the log vectorization and detection phases. Contrastive learning is used to train the vectorization and diagnostic models by aggregating and distinguishing classes in mathematical space, resulting in a more robust vectorization model and better generalization of the generated template vector data. This also improves the feature extraction ability of the diagnostic model and improves the final anomaly classification results. Experimental results show that our method improves the handling of unstable log data in anomaly detection and outperforms the baseline on HDFS and BGL datasets in terms of experimental performance.