Semi-supervised semantic segmentation focuses on the exploration of a small amount of labeled data and a large amount of unlabeled data, which is more in line with the demands of real-world image understanding applications. However, it is still hindered by the inability to fully and effectively leverage unlabeled images. In this paper, we reveal that cross-window consistency (CWC) is helpful in comprehensively extracting auxiliary supervision from unlabeled data. Additionally, we propose a novel CWC-driven progressive learning framework to optimize the deep network by mining weak-to-strong constraints from massive unlabeled data. More specifically, this paper presents a biased cross-window consistency (BCC) loss with an importance factor, which helps the deep network explicitly constrain confidence maps from overlapping regions in different windows to maintain semantic consistency with larger contexts. In addition, we propose a dynamic pseudo-label memory bank (DPM) to provide high-consistency and high-reliability pseudo-labels to further optimize the network. Extensive experiments on three representative datasets of urban views, medical scenarios, and satellite scenes with consistent performance gain demonstrate the superiority of our framework. Our code is released at https://jack-bo1220.github.io/project/CWC.html.
IEEE transactions on image processing : a publication of the IEEE Signal Processing Society. 2024:33:5219-5231. doi: 10.1109/TIP.2024.3458854 Q113.72024
Progressive Learning With Cross-Window Consistency for Semi-Supervised Semantic Segmentation
基于跨窗口一致性的半监督语义分割的渐进学习方法 翻译改进
作者单位 +展开
作者单位
DOI: 10.1109/TIP.2024.3458854 PMID: 39288046
摘要 中英对照阅读
Keywords:progressive learning; cross-window consistency
半监督语义分割侧重于探索少量标记数据和大量未标记数据的利用,这更符合现实世界图像理解应用的需求。然而,它仍然受到无法充分有效利用未标记图像的限制。在本文中,我们揭示了跨窗口一致性(Cross-Window Consistency, CWC)有助于全面从未标记数据中提取辅助监督信息。此外,我们提出了一种新的由CWC驱动的渐进式学习框架,通过挖掘大量未标记数据中的弱到强约束来优化深度网络。具体来说,本文介绍了一种带有重要性因子的偏置跨窗口一致性(Biased Cross-Window Consistency, BCC)损失函数,它有助于深度网络明确地从不同窗口中重叠区域的自信图保持语义与更大上下文的一致性。此外,我们提出了一种动态伪标签记忆库(Dynamic Pseudo-Label Memory Bank, DPM),以提供高一致性和高可靠性的伪标签来进一步优化网络。在三个代表性的城市视图、医疗场景和卫星图像数据集上进行的大量实验表明了我们框架的一致性能提升,展示了其优越性。我们的代码发布在 https://jack-bo1220.github.io/project/CWC.html。
相关内容
-
Progressive Learning of Category-Consistent Multi-Granularity Features for Fine-Grained Visual Classification
细粒度视觉分类的类别一致性多粒度特征的渐进式学习方法研究
IEEE transactions on pattern analysis and machine intelligence. 2022 Dec;44(12):9521-9535.
-
Neuronal Population Reconstruction From Ultra-Scale Optical Microscopy Images via Progressive Learning
通过渐进学习从超大规模光学显微镜图像中重构神经元群体
IEEE transactions on medical imaging. 2020 Dec;39(12):4034-4046.
-
Progressive Learning Strategy for Few-Shot Class-Incremental Learning
面向少量样本类增量学习的渐进式学习策略
IEEE transactions on cybernetics. 2025 Jan 22:PP.
-
Perturbed Progressive Learning for Semisupervised Defect Segmentation
半监督缺陷分割的扰动渐进学习方法
IEEE transactions on neural networks and learning systems. 2024 May;35(5):6118-6132.
-
Blind VQA on 360° Video via Progressively Learning from Pixels, Frames and Video
基于逐级学习像素、帧和视频的360度视频盲式VQA
IEEE transactions on image processing : a publication of the IEEE Signal Processing Society. 2022 Dec 7:PP.
-
Real-Time Progressive Learning: Accumulate Knowledge From Control With Neural-Network-Based Selective Memory
基于神经网络的选择性记忆的实时渐进学习:从控制中积累知识
IEEE transactions on neural networks and learning systems. 2024 Dec 30:PP.
-
Progressive learning and its application to robot impedance learning
连续学习及其在机器人阻抗学习中的应用
IEEE transactions on neural networks. 1996;7(4):941-52.
-
PET image reconstruction with deep progressive learning
基于深度渐进学习的正电子发射断层扫描图像重建技术
Physics in medicine and biology. 2021 May 14;66(10).