Text Semantic-Guided Dynamic Curriculum Learning for Class-Incremental Object Detection
-
摘要: 类增量目标检测旨在持续学习新类别的同时,保持对已学习旧类别的检测能力。现有方法存在目标类别判别能力不足、旧类伪标签可靠性低以及新旧知识难以平衡等问题。为此,提出一种文本语义引导的动态课程式学习类增量目标检测方法。首先构建基于语义先验的判别模块,利用类别文本原型与视觉语义特征的相似度完成分类,增强模型类别判别能力;其次提出视觉文本辅助的伪标签筛选模块,通过视觉语言问答判断教师预测框的旧类归属性,仅保留语义一致且无冲突的旧类预测参与回放监督,减少低质量伪标签干扰;最后设计动态自适应课程式蒸馏策略,根据新类检测损失变化自适应触发旧类蒸馏,并逐步提升蒸馏权重,实现新类学习与旧类知识保持的平滑过渡。在单阶段增量场景中,所提方法在COCO数据集70+10和40+40的划分下,AP较基线分别提升1.8%和2.1%;在HiFSOD-Bird数据集
1074 +358和716+716的划分下,AP较基线分别提升2.4%和2.2%,并且该方法在多阶段增量场景中也优于多种主流方法。实验结果表明,所提方法能够有效缓解灾难性遗忘,在粗粒度和细粒度增量场景中均具有较好的检测精度、稳定性和鲁棒性。Abstract:Objective Class-incremental object detection requires a detector to learn new categories while retaining its ability to recognize previously learned ones, typically without access to historical training data. Existing approaches mainly rely on knowledge distillation and pseudo-label replay, but still face three limitations: visual classification heads lack stable semantic priors, confidence-based replay may retain semantically incorrect teacher predictions, and fixed distillation schedules cannot adapt to the learning progress of new classes. These limitations lead to category confusion, noisy supervision, and conflicts between new-class learning and old-class preservation. To address these issues, we develop TD-DETR, a text semantic-guided dynamic curriculum learning framework for class-incremental object detection. Methods TD-DETR builds on Deformable DETR and extends the CC-DETR baseline with three complementary components. First, the semantic prior-based discrimination module (SPDM) encodes multiple prompts for each category using a frozen CLIP text encoder and averages their representations to obtain category-level text prototypes. Detection queries are projected into the CLIP semantic space and classified according to their similarity to these prototypes. This provides stable semantic constraints for distinguishing old and new categories. Second, the vision-text assisted pseudo-label filtering module (VTPF) refines old-class predictions generated by the teacher detector. For each candidate box, its image region and predicted category are used to construct a visual question. A frozen vision-language model then determines whether the region is semantically consistent with the predicted old category. Only consistent candidates are retained, after which IOU-based conflict suppression removes boxes that overlap with current ground-truth annotations. Third, the dynamic adaptive curriculum distillation strategy (DACD) adjusts old-class distillation according to the new-class classification loss. The detector initially focuses on learning current categories. Distillation is activated when the loss falls to a predefined level or a maximum waiting epoch is reached, and its weight is subsequently increased in a linear manner. This curriculum enables a gradual transition from new-class acquisition to the joint optimization of new- and old-class knowledge. Results and Discussions We evaluate TD-DETR on the coarse-grained MS COCO 2017 dataset and the fine-grained HiFSOD-Bird dataset. MS COCO 2017 uses the single-stage splits 70+10 and 40+40 and the multi-stage splits 40+20+20 and 40+10+10+10+10. HiFSOD-Bird contains 176,350 images from 1,432 bird categories and is evaluated under the single-stage splits 1074 +358 and 716+716 and the multi-stage splits 716+358+358 and 716+179+179+179+179. On MS COCO 2017, TD-DETR achieves AP scores of 42.7% and 41.2% under the 70+10 and 40+40 settings, outperforming the CC-DETR baseline by 1.8 and 2.1 percentage points, respectively. On HiFSOD-Bird, it obtains 46.5% and 47.2% AP under the1074 +358 and 716+716 settings, improving upon CC-DETR by 2.4 and 2.2 percentage points. TD-DETR also maintains higher accuracy and a slower performance decline throughout the multi-stage experiments. Ablation studies confirm the complementary effects of the three components. On the MS COCO 2017 70+10 split, SPDM, VTPF, and DACD progressively increase AP from 40.9% to 41.4%, 42.1%, and 42.7%. On the HiFSOD-Bird1074 +358 split, AP increases from 44.1% to 44.9%, 46.0%, and 46.5%, respectively. These results show that semantic priors improve category discrimination, vision-language verification enhances pseudo-label reliability, and curriculum distillation better balances new-class learning and old-class preservation.Conclusions We present TD-DETR to address category confusion, unreliable old-class pseudo labels, and conflicting training objectives in class-incremental object detection. By combining semantic prior-based discrimination, vision-text assisted pseudo-label filtering, and adaptive curriculum distillation, TD-DETR improves both new-class learning and old-class knowledge retention. Experiments on MS COCO 2017 and HiFSOD-Bird demonstrate consistent gains across coarse-grained, fine-grained, single-stage, and multi-stage settings, indicating that text semantics provide effective guidance for alleviating catastrophic forgetting and improving incremental detection stability. -
Key words:
- Object detection /
- Text semantic guidance /
- Class-increment /
- Curriculum learning
-
表 1 COCO数据集对比实验结果(%)
设置 方法 检测器 AP AP50 AP75 APS APM APL 70+10 RILOD19 GFLv1 24.5 37.9 25.7 14.2 27.4 33.5 SID21 GFLv1 32.8 49.0 35.0 17.1 36.9 44.5 ERD23 GFLv1 34.9 51.9 37.4 18.7 38.8 45.5 IOR25 GFLv1 35.6 49.3 36.6 21.4 38.9 42.4 CL-DETR23 Deformable DETR 39.0 56.8 42.4 22.6 42.2 51.7 DiffKA25 Deformable DETR 40.3 57.6 44.1 22.9 44.0 53.3 CC-DETR26 Deformable DETR 40.9 58.8 44.3 23.9 43.7 53.9 DyQ-DETR25 Deformable DETR 41.9 60.1 45.8 24.1 45.3 55.8 IAQD25 Deformable DETR 42.2 60.5 46.2 25.1 45.2 55.6 BPF24 Faster R-CNN 36.2 56.8 38.9 - - - RGR25 Faster R-CNN 36.6 56.6 39.6 - - - Our(TD-DETR) Deformable DETR 42.7 60.6 45.6 26.6 45.2 56.4 40+40 RILOD19 GFLv1 29.9 45.0 32.0 15.8 33.0 40.5 SID21 GFLv1 34.0 51.4 36.3 18.4 38.4 44.9 ERD23 GFLv1 36.9 54.5 39.6 21.3 40.4 47.5 IOR25 GFLv1 35.1 49.9 37.4 20.8 37.8 44.6 CL-DETR23 Deformable DETR 36.7 54.3 39.0 20.2 39.8 50.1 DiffKA25 Deformable DETR 37.2 55.1 39.8 19.4 40.0 50.3 CC-DETR26 Deformable DETR 39.1 56.6 42.1 21.6 40.7 53.8 DyQ-DETR25 Deformable DETR 39.7 57.5 43.0 21.6 42.9 53.8 IAQD25 Deformable DETR 39.9 57.7 43.4 22.9 43.0 52.2 BPF24 Faster R-CNN 34.4 54.3 37.3 - - - RGR25 Faster R-CNN 35.6 56.0 38.2 - - - Our(TD-DETR) Deformable DETR 41.2 59.1 44.6 24.5 44.3 54.0 注:方法上角标为所提模型发表年限,如CL-DETR23为2023年发表 表 2 COCO数据集多阶段对比实验结果(%)
方法 40+20+20 40+10+10+10+10 1 2 1 2 3 4 SID21 34.0 23.8 34.6 24.1 14.6 12.6 BPF24 34.2 23.4 34.7 24.1 14.6 12.7 IOR25 36.1 32.5 36.4 30.8 26.2 20.7 ERD23 36.7 32.4 36.4 30.8 26.2 20.7 DiffKA25 36.8 36.1 36.6 32.7 31.6 29.6 CL-DETR23 37.2 35.3 36.5 33.4 30.8 28.1 CC-DETR26 40.7 38.2 41.3 38.7 35.4 33.6 Our(TD-DETR) 43.4 40.1 42.8 39.3 36.7 34.8 表 3 HiFSOD-Bird数据集对比实验结果(%)
设置 方法 检测器 AP AP50 AP75 AR AR50 AR75 1074 +358IOR25 GFLv1 38.7 48.8 45.6 64.6 64.9 64.9 BPF24 Faster R-CNN 39.4 49.8 46.6 65.2 65.7 65.7 CL-DETR23 Deformable DETR 41.8 52.9 49.8 71.5 73.2 73.3 CC-DETR26 Deformable DETR 44.1 55.2 52.1 71.6 73.3 73.3 Our(TD-DETR) Deformable DETR 46.5 59.4 55.9 72.9 74.5 74.5 716+716 IOR25 GFLv1 38.4 49.0 45.8 64.5 65.1 65.1 BPF24 Faster R-CNN 40.9 51.5 48.0 66.6 67.1 67.1 CL-DETR23 Deformable DETR 43.0 54.7 51.2 68.9 70.3 70.3 CC-DETR26 Deformable DETR 45.0 57.1 53.6 71.0 72.8 72.8 Our(TD-DETR) Deformable DETR 47.2 60.3 56.4 73.6 75.5 75.5 表 4 HiFSOD-Bird数据集多阶段对比实验结果(%)
方法 716+358+358 716+179+179+179+179 1 2 1 2 3 4 BPF24 37.7 31.2 38.1 31.4 25.3 19.8 IOR25 37.2 30.6 38.1 31.4 25.3 19.8 CL-DETR23 38.9 31.4 38.3 32.0 26.7 21.6 CC-DETR26 41.5 33.1 41.4 33.7 28.6 24.4 Our(TD-DETR) 43.8 36.1 43.5 37.1 32.4 29.1 表 5 COCO蒸馏介入早晚实验对比(%)
状态 λ emax AP APold APnew 介入早 0.7 15 42.3 44.5 31.5 介入晚 0.3 25 42.0 44.1 32.8 介入适中 0.5 20 42.7 44.4 32.2 表 6 COCO数据集蒸馏权重增加缓急实验对比(%)
状态 r ωmax AP APold APnew 权重增长较慢 12 3 42.2 43.4 31.7 权重增长较快 8 5 42.4 43.4 30.6 权重增长适中 10 4 42.7 44.4 32.2 表 7 COCO数据集IOU阈值实验对比(%)
δ AP AP50 AP75 APS APM APL 0.6 41.8 59.9 45.0 25.8 44.9 55.1 0.7 42.7 60.6 45.6 26.6 45.2 56.4 0.8 42.3 60.1 45.2 26.2 45.0 55.8 表 10 COCO数据集文本模板不同数量对比(%)
模板数 编码文本总数 生成时间/s AP APold APnew 1 92 0.21 41.9 43.0 31.7 3 276 0.28 42.7 44.4 32.2 5 460 0.35 42.8 44.5 32.0 表 8 COCO数据集消融实验结果(%)
实验序号 方法 SPDM VTPF DACD AP APold APnew 1 Baseline(CC-DETR) 40.9 42.5 31.0 2 +SPDM √ 41.4 42.7 31.8 3 +VTPF √ √ 42.1 43.9 31.9 4 Ours(TD-DETR) √ √ √ 42.7 44.4 32.2 表 9 HiFSOD-Bird数据集消融实验结果(%)
实验序号 方法 SPDM VTPF DACD AP APold APnew 1 Baseline(CC-DETR) 44.1 43.3 52.3 2 +SPDM √ 44.9 43.8 54.1 3 +VTPF √ √ 46.0 44.7 54.8 4 Ours(TD-DETR) √ √ √ 46.5 45.2 55.6 -
[1] 梁燕, 杨会林, 邵凯. 自适应特征选择的车路协同3D目标检测方案[J]. 电子与信息学报, 2025, 47(12): 5214–5225. doi: 10.11999/JEIT250601.LIANG Yan, YANG Huilin, and SHAO Kai. A vehicle-infrastructure cooperative 3D object detection scheme based on adaptive feature selection[J]. Journal of Electronics & Information Technology, 2025, 47(12): 5214–5225. doi: 10.11999/JEIT250601. [2] 吕晓华, 魏铭辰, 刘立波. 基于位置可学习视觉中心机制的零售商品检测方法[J]. 物联网学报, 2023, 7(4): 142–152. doi: 10.11959/j.issn.2096-3750.2023.00366.LYU Xiaohua, WEI Mingchen, and LIU Libo. Retail commodity detection method based on location learnable visual center mechanism[J]. Chinese Journal on Internet of Things, 2023, 7(4): 142–152. doi: 10.11959/j.issn.2096-3750.2023.00366. [3] 翁星星, 庞超, 许博文, 等. 面向遥感图像解译的增量深度学习[J]. 电子与信息学报, 2024, 46(10): 3979–4001. doi: 10.11999/JEIT240172.WENG Xingxing, PANG Chao, XU Bowen, et al. Incremental deep learning for remote sensing image interpretation[J]. Journal of Electronics & Information Technology, 2024, 46(10): 3979–4001. doi: 10.11999/JEIT240172. [4] 李斌, 崔宗勇, 汪浩瀚, 等. SAR目标增量识别中基于最大化非重合体积的样例挑选方法[J]. 电子与信息学报, 2024, 46(10): 3918–3927. doi: 10.11999/JEIT240217.LI Bin, CUI Zongyong, WANG Haohan, et al. Exemplar selection based on maximizing non-overlapping volume in SAR target incremental recognition[J]. Journal of Electronics & Information Technology, 2024, 46(10): 3918–3927. doi: 10.11999/JEIT240217. [5] 张东阳, 陆子轩, 刘军民, 等. 深度模型的持续学习综述: 理论、方法和应用[J]. 电子与信息学报, 2024, 46(10): 3849–3878. doi: 10.11999/JEIT240095.ZHANG Dongyang, LU Zixuan, LIU Junmin, et al. A survey of continual learning with deep networks: Theory, method and application[J]. Journal of Electronics & Information Technology, 2024, 46(10): 3849–3878. doi: 10.11999/JEIT240095. [6] SHMELKOV K, SCHMID C, and ALAHARI K. Incremental learning of object detectors without catastrophic forgetting[C]. Proceedings of 2017 IEEE International Conference on Computer Vision, Venice, Italy, 2017: 3420–3429. doi: 10.1109/ICCV.2017.368. [7] PENG Can, ZHAO Kun, and LOVELL B C. Faster ILOD: Incremental learning for object detectors based on faster RCNN[J]. Pattern Recognition Letters, 2020, 140: 109–115. doi: 10.1016/j.patrec.2020.09.030. [8] FENG Tao, WANG Mang, and YUAN Hangjie. Overcoming catastrophic forgetting in incremental object detection via elastic response distillation[C]. Proceedings of 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, USA, 2022: 9427–9436. doi: 10.1109/CVPR52688.2022.00921. [9] PENG Can, ZHAO Kun, MAKSOUD S, et al. DIODE: Dilatable incremental object detection[J]. Pattern Recognition, 2023, 136: 109244. doi: 10.1016/j.patcog.2022.109244. [10] ZHU Xizhou, SU Weijie, LU Lewei, et al. Deformable DETR: Deformable transformers for end-to-end object detection[C]. Proceedings of the 9th International Conference on Learning Representations, 2021. (查阅网上资料, 未找到对应的出版地信息, 请确认). [11] LIU Yaoyao, SCHIELE B, VEDALDI A, et al. Continual detection transformer for incremental object detection[C]. Proceedings of 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Vancouver, Canada, 2023: 23799–23808. doi: 10.1109/CVPR52729.2023.02279. [12] ZHANG Jichuan, LI Wei, CHENG Shuang, et al. Dynamic object queries for transformer-based incremental object detection[C]. Proceedings of 2025 IEEE International Conference on Acoustics, Speech and Signal Processing, Hyderabad, India, 2025: 1–5. doi: 10.1109/ICASSP49660.2025.10888317. [13] RADFORD A, KIM J W, HALLACY C, et al. Learning transferable visual models from natural language supervision[C]. Proceedings of the 38th International Conference on Machine Learning, 2021: 8748–8763. (查阅网上资料, 未找到对应的出版地信息, 请确认). [14] LI Dawei, TASCI S, GHOSH S, et al. RILOD: Near real-time incremental learning for object detection at the edge[C]. Proceedings of the 4th ACM/IEEE Symposium on Edge Computing, Arlington, USA, 2019: 113–126. doi: 10.1145/3318216.3363317. [15] PENG Can, ZHAO Kun, MAKSOUD S, et al. SID: Incremental learning for anchor-free object detection via selective and inter-related distillation[J]. Computer Vision and Image Understanding, 2021, 210: 103229. doi: 10.1016/j.cviu.2021.103229. [16] BENGIO Y, LOURADOUR J, COLLOBERT R, et al. Curriculum learning[C]. Proceedings of the 26th Annual International Conference on Machine Learning, Montreal, Canada, 2009: 41–48. doi: 10.1145/1553374.1553380. [17] ZHAO Yian, LV Wenyu, XU Shangliang, et al. DETRs beat YOLOs on real-time object detection[C]. Proceedings of 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, USA, 2024: 16965–16974. doi: 10.1109/CVPR52733.2024.01605. [18] 高鹏, 张若楠, 刘立波. CC-DETR: 基于课程式学习蒸馏的类增量目标检测方法[J]. 物联网学报, 2026. doi: 10.11959/j.issn.2096-3750.WLW25193.GAO Peng, ZHANG Ruonan, and LIU Libo. CC-DETR: Class incremental object detection algorithm based on curriculum learning distillation[J]. Chinese Journal on Internet of Things, 2026. doi: 10.11959/j.issn.2096-3750.WLW25193. [19] LIU Shilong, ZENG Zhaoyang, REN Tianhe, et al. Grounding DINO: Marrying DINO with grounded pre-training for open-set object detection[C]. Proceedings of the 18th European Conference on Computer Vision, Milan, Italy, 2025. doi: 10.1007/978-3-031-72970-6_3. [20] 才华, 冉越, 付强, 等. 多粒度文本感知分层特征交互的视觉定位方法[J]. 电子与信息学报, 2025, 47(11): 4594–4605. doi: 10.11999/JEIT250387.CAI Hua, RAN Yue, FU Qiang, et al. Multi-granularity text perception and hierarchical feature interaction method for visual grounding[J]. Journal of Electronics & Information Technology, 2025, 47(11): 4594–4605. doi: 10.11999/JEIT250387. [21] PENG Yansong, LI Hebei, WU Peixi, et al. D-FINE: Redefine regression task in DETRs as fine-grained distribution refinement[C]. Proceedings of the International Conference on Learning Representations, Singapore, Singapore, 2025. (查阅网上资料, 无法确认出版地信息是否正确). [22] MO Qijie, GAO Yipeng, FU Shenghao, et al. Bridge past and future: Overcoming information asymmetry in incremental object detection[C]. Proceedings of the 18th European Conference on Computer Vision, Milan, Italy, 2025: 463–480. doi: 10.1007/978-3-031-72640-8_26. [23] MASANA M, LIU Xialei, TWARDOWSKI B, et al. Class-incremental learning: Survey and performance evaluation on image classification[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2023, 45(5): 5513–5533. doi: 10.1109/TPAMI.2022.3213473. [24] LIN T Y, MAIRE M, BELONGIE S, et al. Microsoft COCO: Common objects in context[C]. Proceedings of the 13th European Conference on Computer Vision, Zurich, Switzerland, 2014: 740–755. doi: 10.1007/978-3-319-10602-1_48. [25] ZHANG Lu, WANG Yang, ZHOU Jiaogen, et al. Hierarchical few-shot object detection: Problem, benchmark and method[C]. Proceedings of the 30th ACM International Conference on Multimedia, Lisbon, Portugal, 2022: 2002–2011. doi: 10.1145/3503161.3548412. [26] AN Zijia, DIAO Boyu, HUANG Libo, et al. IOR: Inversed objects replay for incremental object detection[C]. Proceedings of 2025 IEEE International Conference on Acoustics, Speech and Signal Processing, Hyderabad, India, 2025: 1–5. doi: 10.1109/ICASSP49660.2025.10888892. [27] MA Mingxiao, ZHU Shunyao, and KANG Guoliang. Index-aligned query distillation for transformer-based incremental object detection[EB/OL]. https://arxiv.org/abs/2508.11339, 2025. [28] ZHANG Shizhou, LV Xueqiang, XING Yinghui, et al. Revisiting generative replay for class incremental object detection[C]. Proceedings of 2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Nashville, USA, 2025: 20340–20349. doi: 10.1109/CVPR52734.2025.01894. [29] WANG Jianzhou, WU Yirui, YUAN Lixin, et al. Diffuse&Refine: Intrinsic knowledge generation and aggregation for incremental object detection[C]. Proceedings of the 34th International Joint Conference on Artificial Intelligence, Jeju, South Korea, 2025: 700. doi: 10.24963/ijcai.2025/700. -
下载: