Research Article
Deep Reinforcement Learning from Drifting Network Environments in Anomaly Detection
@INPROCEEDINGS{10.1007/978-3-031-94445-1_17, author={Junli Zheng and Shaobing Wang and Zhicheng Xu and Xiaoli Zhang and Hongbing Cheng}, title={Deep Reinforcement Learning from Drifting Network Environments in Anomaly Detection}, proceedings={Security and Privacy in Communication Networks. 20th EAI International Conference, SecureComm 2024, Dubai, United Arab Emirates, October 28--30, 2024, Proceedings, Part I}, proceedings_a={SECURECOMM}, year={2026}, month={9}, keywords={Network intrusion detection system Security Reinforcement learning Machine learning}, doi={10.1007/978-3-031-94445-1_17} }- Junli Zheng
Shaobing Wang
Zhicheng Xu
Xiaoli Zhang
Hongbing Cheng
Year: 2026
Deep Reinforcement Learning from Drifting Network Environments in Anomaly Detection
SECURECOMM
Springer
DOI: 10.1007/978-3-031-94445-1_17
Abstract
Recently, we have witnessed the widespread adoption of Machine Learning (ML) techniques in the domain of network intrusion detection. However, most ML-based network intrusion detection systems (NIDSes) operate under the closed-world assumption where the training and test datasets consist of uniformly distributed normal or abnormal samples. This assumption often fails to hold in real-world scenarios, where both malware and normal behaviors evolve over time, leading to dynamic changes in network distribution (known as ). In the paper, we design a deep reinforcement learning (RL) based network intrusion detection system capable of automatically adapting to concept drift in the open world. Particularly, we introduce a quality-driven drift detector to lightweight select and label drift samples. Then, our method can effectively adapt both normality and abnormality shifts via a sampling function within a dynamic simulation environment. We validate the effectiveness of our method using two public datasets exhibiting real-world drift. Experimental results demonstrate that our method achieves an F1 score of 0.93, outperforming the state-of-the-art(SOTA) methods.