ENIDrift: A Fast and Adaptive Ensemble System for Network Intrusion Detection under Real-World Drift

No ratings

Presented at ACSAC 2022 by

Xian Wang

doi.org

Machine Learning (ML) techniques have been widely applied for network intrusion detection. However, existing ML-based network intrusion detection systems (NIDSs) suffer from fundamental limitations that hinder them from being deployed in the real world. They consider a narrow scope rather than real-world drift that involves dynamically distributed network packets and well-crafted ML attacks. Besides, they pose high runtime overhead and have low processing speed. In this paper, we solve the limitations and design ENIDrift, a fast and adaptive ensemble system for real-world network intrusion detection. ENIDrift employs iP2V, a novel incremental feature extraction method based on network packet fields, which adopts a simple three-layer neural network with relatively lightweight computation and achieves high efficiency. ENIDrift uses a stable sub-classifier generation module that constructs new sub-classifiers based on the stability and accuracy of incoming data chunks, and its training time is reduced from to . We extend the threat model and place experiments in real-world settings. We make the first real-world drift dataset, RWDIDS, which contains intense and different drifts for NIDS. Our extensive evaluation under real-world drift demonstrates that ENIDrift significantly outperforms the state-of-the-art solutions by up to 69.78% of F1 and reduces running time by 87.6%. ENIDrift achieves a 100% F1 against our adversarial attack and is adaptive to various real-world drifts. Our field test also shows ENIDrift functions well even with delayed, inadequate training data, which is practical for real-world usage.