Accurate segmentation of camouflage objects in aerial imagery is vital for improving the efficiency of UAV-based reconnaissance and rescue missions.However,camouflage object segmentation is increasingly challenging du...Accurate segmentation of camouflage objects in aerial imagery is vital for improving the efficiency of UAV-based reconnaissance and rescue missions.However,camouflage object segmentation is increasingly challenging due to advances in both camouflage materials and biological mimicry.Although multispectral-RGB based technology shows promise,conventional dual-aperture multispectral-RGB imaging systems are constrained by imprecise and time-consuming registration and fusion across different modalities,limiting their performance.Here,we propose the Reconstructed Multispectral-RGB Fusion Network(RMRF-Net),which reconstructs RGB images into multispectral ones,enabling efficient multimodal segmentation using only an RGB camera.Specifically,RMRF-Net employs a divergentsimilarity feature correction strategy to minimize reconstruction errors and includes an efficient boundary-aware decoder to enhance object contours.Notably,we establish the first real-world aerial multispectral-RGB semantic segmentation of camouflage objects dataset,including 11 object categories.Experimental results demonstrate that RMRF-Net outperforms existing methods,achieving 17.38 FPS on the NVIDIA Jetson AGX Orin,with only a 0.96%drop in mIoU compared to the RTX 3090,showing its practical applicability in multimodal remote sensing.展开更多
With the development of underwater sonar detection technology,simultaneous localization and mapping(SLAM)approach has attracted much attention in underwater navigation field in recent years.But the weak detection abil...With the development of underwater sonar detection technology,simultaneous localization and mapping(SLAM)approach has attracted much attention in underwater navigation field in recent years.But the weak detection ability of a single vehicle limits the SLAM performance in wide areas.Thereby,cooperative SLAM using multiple vehicles has become an important research direction.The key factor of cooperative SLAM is timely and efficient sonar image transmission among underwater vehicles.However,the limited bandwidth of underwater acoustic channels contradicts a large amount of sonar image data.It is essential to compress the images before transmission.Recently,deep neural networks have great value in image compression by virtue of the powerful learning ability of neural networks,but the existing sonar image compression methods based on neural network usually focus on the pixel-level information without the semantic-level information.In this paper,we propose a novel underwater acoustic transmission scheme called UAT-SSIC that includes semantic segmentation-based sonar image compression(SSIC)framework and the joint source-channel codec,to improve the accuracy of the semantic information of the reconstructed sonar image at the receiver.The SSIC framework consists of Auto-Encoder structure-based sonar image compression network,which is measured by a semantic segmentation network's residual.Considering that sonar images have the characteristics of blurred target edges,the semantic segmentation network used a special dilated convolution neural network(DiCNN)to enhance segmentation accuracy by expanding the range of receptive fields.The joint source-channel codec with unequal error protection is proposed that adjusts the power level of the transmitted data,which deal with sonar image transmission error caused by the serious underwater acoustic channel.Experiment results demonstrate that our method preserves more semantic information,with advantages over existing methods at the same compression ratio.It also improves the error tolerance and packet loss resistance of transmission.展开更多
Semantic segmentation is a crucial step for document understanding.In this paper,an NVIDIA Jetson Nano-based platform is applied for implementing semantic segmentation for teaching artificial intelligence concepts and...Semantic segmentation is a crucial step for document understanding.In this paper,an NVIDIA Jetson Nano-based platform is applied for implementing semantic segmentation for teaching artificial intelligence concepts and programming.To extract semantic structures from document images,we present an end-to-end dilated convolution network architecture.Dilated convolutions have well-known advantages for extracting multi-scale context information without losing spatial resolution.Our model utilizes dilated convolutions with residual network to represent the image features and predicting pixel labels.The convolution part works as feature extractor to obtain multidimensional and hierarchical image features.The consecutive deconvolution is used for producing full resolution segmentation prediction.The probability of each pixel decides its predefined semantic class label.To understand segmentation granularity,we compare performances at three different levels.From fine grained class to coarse class levels,the proposed dilated convolution network architecture is evaluated on three document datasets.The experimental results have shown that both semantic data distribution imbalance and network depth are import factors that influence the document’s semantic segmentation performances.The research is aimed at offering an education resource for teaching artificial intelligence concepts and techniques.展开更多
针对遥感地物建筑物图像目标尺度差异大、样本空间分布不均衡、地物边界模糊、场景区域跨度大所导致的分割效果不佳问题,本文提出一种融合动态特征增强高精度遥感建筑物分割算法。首先,构建New_GhostNetV2网络,利用自适应上下文感知卷积...针对遥感地物建筑物图像目标尺度差异大、样本空间分布不均衡、地物边界模糊、场景区域跨度大所导致的分割效果不佳问题,本文提出一种融合动态特征增强高精度遥感建筑物分割算法。首先,构建New_GhostNetV2网络,利用自适应上下文感知卷积,增强算法对样本空间特征的捕捉能力。其次,采用Ghost Convolution结合跳跃连接和特征分支策略设计多层级信息增强模块,增强特征整合。随后引入级联注意力CGA(cascaded group attention),通过组内独立注意力计算,加强模型对多样化地物形态的适应性。最后,通过动态深度特征增强器构造特征融合模块,进一步加强模型捕获能力。在WHU数据集上实验结果表明:改进算法较基线模型F1-Score提高8.57%,mIoU提高12.48%,与其他主流语义分割模型相比,改进DeepLabv3+具有更好的分割精度。展开更多
针对传统视觉SLAM(simultaneous localization and mapping)在动态环境下定位精度较低、稳健性较差、结合深度学习后实时性较差及无法构建稠密地图的问题,本文提出了一种基于ORB-SLAM3的改进算法。首先,采用轻量化SegFormer语义分割网络...针对传统视觉SLAM(simultaneous localization and mapping)在动态环境下定位精度较低、稳健性较差、结合深度学习后实时性较差及无法构建稠密地图的问题,本文提出了一种基于ORB-SLAM3的改进算法。首先,采用轻量化SegFormer语义分割网络,对图像中存在的动态物体进行识别后,添加掩膜图像自适应膨胀方法,根据特征点数自动调整掩膜膨胀范围,更有效地保留静态特征点及去除潜在动态特征点;然后,改进词袋模型,提升算法的加载和匹配速度;最后,添加稠密建图线程,根据掩膜信息和关键帧,构建去除动态特征后的稠密点云地图。试验结果表明,该算法在动态场景下能够有效地剔除动态物体特征点,提高了系统的定位精度和稳健性,平均处理速度为20帧/s,基本满足实时运行的要求。展开更多
基金National Natural Science Foundation of China(Grant Nos.62005049 and 62072110)Natural Science Foundation of Fujian Province(Grant No.2020J01451).
文摘Accurate segmentation of camouflage objects in aerial imagery is vital for improving the efficiency of UAV-based reconnaissance and rescue missions.However,camouflage object segmentation is increasingly challenging due to advances in both camouflage materials and biological mimicry.Although multispectral-RGB based technology shows promise,conventional dual-aperture multispectral-RGB imaging systems are constrained by imprecise and time-consuming registration and fusion across different modalities,limiting their performance.Here,we propose the Reconstructed Multispectral-RGB Fusion Network(RMRF-Net),which reconstructs RGB images into multispectral ones,enabling efficient multimodal segmentation using only an RGB camera.Specifically,RMRF-Net employs a divergentsimilarity feature correction strategy to minimize reconstruction errors and includes an efficient boundary-aware decoder to enhance object contours.Notably,we establish the first real-world aerial multispectral-RGB semantic segmentation of camouflage objects dataset,including 11 object categories.Experimental results demonstrate that RMRF-Net outperforms existing methods,achieving 17.38 FPS on the NVIDIA Jetson AGX Orin,with only a 0.96%drop in mIoU compared to the RTX 3090,showing its practical applicability in multimodal remote sensing.
基金supported in part by the Tianjin Technology Innovation Guidance Special Fund Project under Grant No.21YDTPJC00850in part by the National Natural Science Foundation of China under Grant No.41906161in part by the Natural Science Foundation of Tianjin under Grant No.21JCQNJC00650。
文摘With the development of underwater sonar detection technology,simultaneous localization and mapping(SLAM)approach has attracted much attention in underwater navigation field in recent years.But the weak detection ability of a single vehicle limits the SLAM performance in wide areas.Thereby,cooperative SLAM using multiple vehicles has become an important research direction.The key factor of cooperative SLAM is timely and efficient sonar image transmission among underwater vehicles.However,the limited bandwidth of underwater acoustic channels contradicts a large amount of sonar image data.It is essential to compress the images before transmission.Recently,deep neural networks have great value in image compression by virtue of the powerful learning ability of neural networks,but the existing sonar image compression methods based on neural network usually focus on the pixel-level information without the semantic-level information.In this paper,we propose a novel underwater acoustic transmission scheme called UAT-SSIC that includes semantic segmentation-based sonar image compression(SSIC)framework and the joint source-channel codec,to improve the accuracy of the semantic information of the reconstructed sonar image at the receiver.The SSIC framework consists of Auto-Encoder structure-based sonar image compression network,which is measured by a semantic segmentation network's residual.Considering that sonar images have the characteristics of blurred target edges,the semantic segmentation network used a special dilated convolution neural network(DiCNN)to enhance segmentation accuracy by expanding the range of receptive fields.The joint source-channel codec with unequal error protection is proposed that adjusts the power level of the transmitted data,which deal with sonar image transmission error caused by the serious underwater acoustic channel.Experiment results demonstrate that our method preserves more semantic information,with advantages over existing methods at the same compression ratio.It also improves the error tolerance and packet loss resistance of transmission.
基金Project(61806107)supported by the National Natural Science Foundation of ChinaProject supported by the Shandong Key Laboratory of Wisdom Mine Information Technology,ChinaProject supported by the Opening Project of State Key Laboratory of Digital Publishing Technology,China。
文摘Semantic segmentation is a crucial step for document understanding.In this paper,an NVIDIA Jetson Nano-based platform is applied for implementing semantic segmentation for teaching artificial intelligence concepts and programming.To extract semantic structures from document images,we present an end-to-end dilated convolution network architecture.Dilated convolutions have well-known advantages for extracting multi-scale context information without losing spatial resolution.Our model utilizes dilated convolutions with residual network to represent the image features and predicting pixel labels.The convolution part works as feature extractor to obtain multidimensional and hierarchical image features.The consecutive deconvolution is used for producing full resolution segmentation prediction.The probability of each pixel decides its predefined semantic class label.To understand segmentation granularity,we compare performances at three different levels.From fine grained class to coarse class levels,the proposed dilated convolution network architecture is evaluated on three document datasets.The experimental results have shown that both semantic data distribution imbalance and network depth are import factors that influence the document’s semantic segmentation performances.The research is aimed at offering an education resource for teaching artificial intelligence concepts and techniques.
文摘针对遥感地物建筑物图像目标尺度差异大、样本空间分布不均衡、地物边界模糊、场景区域跨度大所导致的分割效果不佳问题,本文提出一种融合动态特征增强高精度遥感建筑物分割算法。首先,构建New_GhostNetV2网络,利用自适应上下文感知卷积,增强算法对样本空间特征的捕捉能力。其次,采用Ghost Convolution结合跳跃连接和特征分支策略设计多层级信息增强模块,增强特征整合。随后引入级联注意力CGA(cascaded group attention),通过组内独立注意力计算,加强模型对多样化地物形态的适应性。最后,通过动态深度特征增强器构造特征融合模块,进一步加强模型捕获能力。在WHU数据集上实验结果表明:改进算法较基线模型F1-Score提高8.57%,mIoU提高12.48%,与其他主流语义分割模型相比,改进DeepLabv3+具有更好的分割精度。
文摘针对传统视觉SLAM(simultaneous localization and mapping)在动态环境下定位精度较低、稳健性较差、结合深度学习后实时性较差及无法构建稠密地图的问题,本文提出了一种基于ORB-SLAM3的改进算法。首先,采用轻量化SegFormer语义分割网络,对图像中存在的动态物体进行识别后,添加掩膜图像自适应膨胀方法,根据特征点数自动调整掩膜膨胀范围,更有效地保留静态特征点及去除潜在动态特征点;然后,改进词袋模型,提升算法的加载和匹配速度;最后,添加稠密建图线程,根据掩膜信息和关键帧,构建去除动态特征后的稠密点云地图。试验结果表明,该算法在动态场景下能够有效地剔除动态物体特征点,提高了系统的定位精度和稳健性,平均处理速度为20帧/s,基本满足实时运行的要求。
