An Anomalous Traffic Detection Method Integrating Flow Data Compression and Self-supervised Graph Learning
-
摘要: 在网络流量规模持续增长、攻击手段日趋复杂的背景下，现有异常流量检测方法面临流特征提取效率低、对标注数据依赖强以及缺乏通信关系上下文建模的问题。为此，该文提出一种融合流数据压缩与图自监督学习的异常流量检测方法SketchGNN。首先，设计基于流数据压缩的自适应特征提取机制，依据流量分布动态调整数据结构，在有限内存开销下高效提取多样化流特征，并显著提升测量精度；之后，构建基于自监督学习的图神经网络模型，通过消息传递机制学习拓扑结构以建模流量上下文，生成对正常流量波动具备鲁棒性的高质量边嵌入，最终结合无监督分类算法实现无标注数据依赖的高效异常检测。实验结果表明，SketchGNN在多种无监督分类器上的检测准确率均不低于95.2%，且相比基线方案在特征测量误差关键指标上降低至少31.0%，验证了所提方法在大规模网络异常流量检测场景中的有效性与可扩展性。Abstract:
Objective As network traffic volumes continue to grow and attack methods become increasingly sophisticated, efficient and intelligent anomalous traffic detection is essential for protecting critical information infrastructure. However, existing detection methods still face substantial challenges in large-scale network environments. On one hand, analyzing raw packet sequences and using deep learning-based end-to-end models incur considerable computational and storage overhead, making them difficult to deploy in line-rate processing scenarios. On the other hand, flow records are usually treated as independent samples, while the topological structure and contextual information of inter-host communications are often ignored. This limitation makes it difficult to detect distributed and correlated threats from a global perspective. In addition, supervised learning methods rely heavily on large amounts of labeled data, which are difficult to obtain in practical deployments and limit generalization to unknown threats. Therefore, an anomalous traffic detection method that supports efficient flow feature extraction under limited resources and enables accurate detection without labeled data is needed. Methods During training, the original communication graph and augmented negative samples are simultaneously input into a graph encoder to learn edge embeddings. The resulting embeddings are then fed into a discriminator. Mutual information scores are estimated by contrasting the edge embeddings of positive and negative samples with a global graph summary. The training objective is to maximize the scores of positive samples and minimize those of negative samples. Through this self-supervised optimization, the encoder parameters are refined to improve the discriminative capability of the edge embeddings. The process is iterated using gradient descent until convergence. After training, the encoder parameters are fixed, and the resulting edge embeddings are used for downstream anomalous traffic detection. During inference, traffic is converted into a communication graph by the feature extractor and then fed into the trained graph encoder to generate the corresponding edge embeddings. A lightweight classifier takes these embeddings as input to perform end-to-end anomalous traffic detection and output the final classification results. Results and Discussions Comprehensive experiments are conducted on four public datasets, namely CAIDA, CIC-IDS2018, UNSW-NB15, and TON-IoT. For feature extraction, under identical memory configurations, the Average Relative Error (ARE) and per-flow Weighted Mean Relative Error (WMRE) of counter features measured by MFSketch-OP are reduced by 31.5% and 31.0%, respectively, compared with the baseline MFSketch with a fixed structure. For bitmap features, the corresponding reductions are 36.1% and 34.9%, respectively ( Fig. 4 ). High throughput is maintained across datasets with different degrees of traffic skewness, with an average throughput of approximately 12 Mpps achieved on CAIDA (Fig. 4(c) ). For detection accuracy, when combined with Principal Component Analysis (PCA), Histogram-Based Outlier Score (HBOS), or Isolation Forest (IF), SketchGNN consistently achieves an accuracy of at least 95.2%, a macro-F1 score of at least 90.1%, and a weighted-F1 score of at least 96.7% on CIC-IDS2018 and UNSW-NB15. These results generally outperform the baseline methods and show more stable performance across datasets (Figs. 5 and6 ). For detection efficiency, HBOS provides high and stable throughput among the three classifiers (Fig. 7(a) ). The end-to-end packet-level equivalent throughput of SketchGNN reaches 640 kpps, approximately 17 times that of Kitsune (37 kpps), and is comparable in magnitude to that of Whisper accelerated by the Data Plane Development Kit (DPDK) (1.3 Mpps) (Fig. 7(b) ). In addition, the performance variation across different datasets remains within 3%, indicating robust generalization to normal traffic fluctuations and diverse flow-level anomalous behaviors.Conclusions To address the high overhead of feature extraction, insufficient use of traffic context, and strong dependence on labeled data in existing anomalous traffic detection methods, SketchGNN, an anomalous traffic detection framework integrating flow data compression with self-supervised graph learning, is proposed. A dynamically configurable sketch, MFSketch, is used to efficiently extract and accurately measure diverse flow features under limited resource constraints. A self-supervised graph neural network is then used to model host communication graphs and learn traffic representations, enabling efficient anomalous traffic detection without labeled data. Experimental results show that MFSketch dynamically optimizes its data structure according to traffic distribution and provides high-throughput and high-precision feature inputs for downstream detection. The edge embeddings generated through self-supervised graph learning achieve higher detection accuracy than the baseline methods when combined with different unsupervised classifiers. In future work, hybrid detection mechanisms that combine Deep Packet Inspection (DPI) with programmable data planes will be explored to further improve the detection of application-layer anomalous traffic. -
表 1 支持的流级统计特征类型
直接捕获的流特征 计算所得的流特征 流数量(以5元组区分) 包长均值 TCP流数量(以5元组区分) 流速率均值(以包数计) UDP流数量(以5元组区分) 流速率均值(以字节数计) 流量大小(以包数计) 流量突增比(当前计数/历史均值) 流量大小(以字节数计) 流量熵值 
下载: 导出CSV
1 MFSketch自适应配置算法
输入：窗口$ t $的输入流量，初始哈希函数数量$ d $，阈值$ {\theta }_{H} $。 输出：优化后的MFSketch。 1. $ d\mathrm{^{cur}}\leftarrow d $, $ {H}_{\text{prev}}\leftarrow 0 $, $ {\text{cnt}}_{\text{stab}}\leftarrow 0 $ 2. FOR window $ t $ DO 3. $ {\mathcal{C}}_{\text{win}}\leftarrow \varnothing $ 4. $ N\leftarrow \displaystyle\sum \nolimits_{f\in {\mathcal{C}}_{\text{win}}}{\mathcal{C}}_{\text{win}}[f] $ 5. $ H(t)\leftarrow -\displaystyle\sum\nolimits_{f}\dfrac{{\mathcal{C}}_{\text{win}}[f]}{N}\ln \dfrac{{\mathcal{C}}_{\text{win}}[f]}{N} $ //计算当前窗口流量熵值 6. $ \Delta H(t)\leftarrow |H(t)-{H}_{\text{prev}}|/{H}_{\text{prev}} $ 7. IF $ \Delta H(t) \gt {\theta }_{H} $ THEN 8. $ {d}^{\rm{op}}\leftarrow \dfrac{\alpha NM}{e(k\alpha N+MF_{}^{\text{res}(k)})} $ //依据定理1计算最优哈希
函数数量9. IF $ {d}^{\rm{op}}\neq {d}^{\text{cur}} $ THEN 10. $ {d}^{\text{cur}}\leftarrow {d}^{\rm{op}} $, $ {\text{cnt}}_{\text{stab}}\leftarrow 0 $ 11. ELSE 12. $ {\text{cnt}}_{\text{stab}}={\text{cnt}}_{\text{stab}}+1 $ 13. IF $ {\text{cnt}}_{\text{stab}}\geq 3 $ THEN //确认当前草图配置是否稳定 14. $ {\text{cnt}}_{\text{stab}}\leftarrow 0 $ 15. $ {H}_{\text{prev}}\leftarrow H(t) $
下载: 导出CSV
2 面向边嵌入的图自监督学习算法
输入：原始图$ G(\mathcal{V},\mathcal{E}) $及其负样本$ \hat{G} $；节点特征${\mathbf{x}}_{v} = \{1,1,\cdots,1\} $,
$ \forall v\in \mathcal{V} $；边特征$ \{{\mathbf{e}}_{uv},\forall uv\in \mathcal{E}\} $；图编码器层数$ L $；权重矩阵
$ {\mathbf{W}}^{l},\forall l\in \{1,2,\cdots,L\} $；非线性激活函数$ \sigma $；聚合函数 AGG；训
练轮数$ K $。输出：优化后的图编码器$ \mathcal{G} $。 16. FOR $ v\in V $ DO 17. $ \mathbf{h}_{v}^{0}\leftarrow {\mathbf{x}}_{v} $ 18. FOR $ l\leftarrow 1 $ to $ L $ DO // 图编码器$ \mathcal{G} $ 19. FOR $ v\in \mathcal{V} $ DO 20. $ \mathbf{h}_{\mathcal{N}(v)}^{l}\leftarrow {\text{AGG}}_{l}(\{\mathbf{e}_{uv}^{l-1},\forall u\in \mathcal{N}(v),uv\in \mathcal{E}\}) $ 21. $ \mathbf{h}_{v}^{l}\leftarrow \sigma \left({\mathbf{W}}^{l}\cdot \text{CONCAT}(\mathbf{h}_{v}^{l-1},\mathbf{h}_{\mathcal{N}(v)}^{l})\right) $ 22. $ {\mathbf{z}}_{v}=\mathbf{h}_{v}^{L} $ 23. FOR $ \mathrm{epoch}\leftarrow1 $ to $ K $ DO 24. FOR $ (u,v)\in \mathcal{E} $ DO 25. $ {\mathbf{z}}_{uv}\leftarrow \text{CONCAT}({\mathbf{z}}_{u},{\mathbf{z}}_{v}) $ // 原始边嵌入 26. $ {\mathbf{\hat{z}}}_{uv}\leftarrow \text{CONCAT}({\mathbf{\hat{z}}}_{u},{\mathbf{\hat{z}}}_{v}) $ // 负样本边嵌入 27. $ \mathbf{\overline{s}}=\sigma \left(\dfrac{1}{|\mathcal{E}|}\displaystyle\sum \nolimits_{(u,v)\in \mathcal{E}}{\mathbf{z}}_{uv}\right) $ // 全局图摘要 28. $ \mathcal{D}({\mathbf{z}}_{uv},\mathbf{\overline{s}})=\sigma ({\mathbf{z}}_{uv}\mathbf{W\overline{s}}) $ 29. $ \mathcal{D}({\mathbf{\hat{z}}}_{uv},\mathbf{\overline{s}})=\sigma ({\mathbf{\hat{z}}}_{uv}\mathbf{W\overline{s}}) $ 30. $ \mathcal{L} = -\dfrac{1}{2|\mathcal{E}|}\displaystyle\sum\nolimits_{(u,v)\in\mathcal{E}}\left[\ln\mathcal{D}({\mathbf{z}}_{uv},\overline{s}) + \ln(1 - \mathcal{D}(\hat{\mathbf{z}}_{uv},\overline{s}))\right] $ 31. Adam 优化，更新图编码器$ \mathcal{G} $参数 32. RETURN $ \mathcal{G} $
下载: 导出CSV
表 2 各类型攻击流量检测结果(TON-IoT数据集)(%)
攻击类型 Precision Recall 宏F1 FPR FNR backdoor 97.88 98.94 98.38 2.13 1.06 ddos 89.10 91.72 90.06 10.26 8.28 dos 82.33 83.12 82.07 13.82 16.88 injection 87.70 91.68 88.56 15.17 8.32 mitm 90.62 94.77 92.59 0.51 5.23 password 80.24 84.15 78.16 31.70 15.85 ransomware 91.21 94.19 91.73 11.51 5.81 scanning 87.78 90.31 88.47 12.30 9.69 xss 78.98 78.87 78.80 14.21 21.13 加权均值 86.93 89.17 87.07 13.80 10.84
下载: 导出CSV
表 3 CIC-IDS2018数据集消融实验(%)
分类算法 原始特征 嵌入表示 ACC 宏F1 加权F1 ACC 宏F1 加权F1 PCA 91.41 71.07 89.52 96.82 92.04 96.73 HBOS 79.38 56.28 80.45 97.74 94.26 97.66 IF 93.29 85.08 93.50 96.96 92.40 96.88 均值 88.03 70.81 87.82 97.17 92.90 97.09
下载: 导出CSV
表 4 UNSW-NB15数据集消融实验(%)
分类算法 原始特征 嵌入表示 ACC 宏F1 加权F1 ACC 宏F1 加权F1 PCA 94.26 68.36 94.71 98.14 88.24 98.17 HBOS 91.95 66.87 93.40 98.72 92.75 98.81 IF 90.13 63.36 92.19 98.38 90.08 98.43 均值 92.11 66.20 93.43 98.41 90.36 98.47
下载: 导出CSV
-
[1] 胡钰林, 喻鑫岚, 高伟, 等. 低时延工业物联网中移动边缘计算的安全性与可靠性联合优化[J]. 电子与信息学报, 2025, 47(10): 3492–3504. doi: 10.11999/JEIT250262.HU Yulin, YU Xinlan, GAO Wei, et al. Security and reliability-optimal offloading for mobile edge computing in low-latency industrial IoT[J]. Journal of Electronics & Information Technology, 2025, 47(10): 3492–3504. doi: 10.11999/JEIT250262. [2] ERLACHER F and DRESSLER F. On high-speed flow-based intrusion detection using Snort-compatible signatures[J]. IEEE Transactions on Dependable and Secure Computing, 2022, 19(1): 495–506. doi: 10.1109/TDSC.2020.2973992. [3] TUDOSI A D, GRAUR A, BALAN D G, et al. Distributed firewall traffic filtering and intrusion detection using Snort on pfSense firewalls with random forest classification[C]. 2023 46th International Conference on Telecommunications and Signal Processing (TSP), Prague, Czech Republic, 2023: 101–104. doi: 10.1109/TSP59544.2023.10197784. [4] RESENDE P A A and DRUMMOND A C. A survey of random forest based methods for intrusion detection systems[J]. ACM Computing Surveys, 2019, 51(3): 48. doi: 10.1145/3178582. [5] MIRSKY Y, DOITSHMAN T, ELOVICI Y, et al. Kitsune: An ensemble of autoencoders for online network intrusion detection[C]. 25th Annual Network and Distributed System Security Symposium, San Diego, USA, 2018https://arxiv.org/abs/1802.09089, 2018. [6] FU Chuanpu, LI Qi, SHEN Meng, et al. Frequency domain feature based robust malicious traffic detection[J]. IEEE/ACM Transactions on Networking, 2023, 31(1): 452–467. doi: 10.1109/TNET.2022.3195871. [7] KHALID M, MOHSIN A R, ALI J, et al. Optimization of recurrent neural networks for high-performance intrusion detection in network traffic[J]. Cluster Computing, 2025, 28(9): 563. doi: 10.1007/s10586-025-05240-0. [8] 顾伟, 行鸿彦, 侯天浩. 基于网络流量时空特征和自适应加权系数的异常流量检测方法[J]. 电子与信息学报, 2024, 46(6): 2647–2654. doi: 10.11999/JEIT230825.GU Wei, XING Hongyan, and HOU Tianhao. Abnormal traffic detection method based on traffic spatial-temporal features and adaptive weighting coefficients[J]. Journal of Electronics & Information Technology, 2024, 46(6): 2647–2654. doi: 10.11999/JEIT230825. [9] HSIEH K, WONG M, SEGARRA S, et al. NetVigil: Robust and low-cost anomaly detection for east-west data center security[C]. 21st USENIX Symposium on Networked Systems Design and Implementation, Santa Clara, USA, 2024: 1771–1789. [10] 尹梓诺, 陈鸿昶, 马海龙, 等. 无监督自适应抽样与改进孪生网络结合的网络流量异常检测方法[J]. 电子与信息学报, 2025, 47(7): 2211–2224. doi: 10.11999/JEIT241115.YIN Zinuo, CHEN Hongchang, MA Hailong, et al. A network traffic anomaly detection method integrating unsupervised adaptive sampling with enhanced Siamese network[J]. Journal of Electronics & Information Technology, 2025, 47(7): 2211–2224. doi: 10.11999/JEIT241115. [11] HAN Hui, YAN Zheng, JING Xuyang, et al. Applications of sketches in network traffic measurement: A survey[J]. Information Fusion, 2022, 82: 58–85. doi: 10.1016/j.inffus.2021.12.007. [12] LI Yuanpeng, NIU Xian, ZHAO Yikai, et al. TitanLog: Hierarchical and elastic logging for high-speed network data stream[J]. IEEE Transactions on Networking, 2026, 34: 1988–2003. doi: 10.1109/TON.2025.3636509. [13] YUAN Ziqi, SUN Qingyun, ZHOU Haoyi, et al. A comprehensive survey on GNN-based anomaly detection: Taxonomy, methods, and the role of large language models[J]. International Journal of Machine Learning and Cybernetics, 2025, 16(7/8): 4407–4432. doi: 10.1007/s13042-024-02516-6. [14] MA Jie, SU Wei, LI Yikun, et al. Synchronizing DDoS detection and mitigation based graph learning with programmable data plane, SDN[J]. Future Generation Computer Systems, 2024, 154: 206–218. doi: 10.1016/j.future.2023.12.033. [15] LO W W, LAYEGHY S, SARHAN M, et al. E-GraphSAGE: A graph neural network based intrusion detection system[J/OL]. https://arxiv.org/abs/2103.16329v1, 2021. doi: 10.48550/arXiv.2103.16329. [16] VELIČKOVIĆ P, FEDUS W, HAMILTON W L, et al. Deep Graph Infomax[C]. 7th International Conference on Learning Representations, New Orleans, USA, 2019. [17] CAVILLE E, LO W W, LAYEGHY S, et al. Anomal-E: A self-supervised network intrusion detection system based on graph neural networks[J]. Knowledge-Based Systems, 2022, 258: 110030. doi: 10.1016/j.knosys.2022.110030. [18] LIU Jiaqian, BASAT R B, WARDT L D, et al. DISCO: A dynamically configurable sketch framework in skewed data streams[C]. 2024 IEEE 40th International Conference on Data Engineering, Utrecht, Netherlands, 2024: 4801–4814. doi: 10.1109/ICDE60146.2024.00365. [19] NGUYEN H and KASHEF R. TS-IDS: Traffic-aware self-supervised learning for IoT network intrusion detection[J]. Knowledge-Based Systems, 2023, 279: 110966. doi: 10.1016/j.knosys.2023.110966. -
下载:
图(9) / 表(6)
计量
- 文章访问数: 289
- HTML全文浏览量: 65
- PDF下载量: 20
- 被引次数: 0
