渐进 CNN-变压器交替重建网络用于高光谱图像重建--以赤潮探测为例

摘要：光谱重建技术从有限的光谱波段中提取丰富的细节信息，从而提高图像质量和分辨率能力。它在无损检测中得到 应用，提高了检测的精度和鲁棒性。目前的研究主要集中在改善卷积神经网络的局部信息感知或使用Transformer建模长 距离依赖关系。然而，这些方法不能有效地整合全局和局部建模信息，导致图像重建的精度差。本文介绍了一种渐进CNN- Transformer交替重建网络（（PCTARN），以交替利用鲁棒卷积注意力和转置Transformer自注意力。提出了一种双通道CNN- 变压器交替重构模块（（ DPCTARM），在各个层次上动态地引入全局-局部动态先验，以便于提取高频和低频特征。这一增 强有效地加强了PCTARN识别有价值信号的能力。为了验证所提出的方法，一个光谱数据集的基础上选择七个赤潮藻类 收集。峰值信噪比（（ PSNR）指标达到34.58 dB，比马翁和MST++等方法至少高出0.44 dB。而参数和FLOPS分别减少了 41.9%和 38.4%以上。由于建议的 PCTARN 的性能不仅取决于图像质量，但也对光谱保真度，光谱检测赤潮的应用进行 了这一目的。从多光谱图像中选取4 个特征波段，利用PCTARN重建得到 20波段高光谱图像。基于重建图像进行物种 识别和细胞浓度检测。结果表明，PCTARN能增强赤潮样品的空间信号和光谱峰差，在种类识别和细胞浓度检测中，识 别准确率达到94.21%，决定系数（R2）为0.9660，分别比4波段多光谱检测提高了11.55%和11.59%。

关键词：光谱重建；CNN-变压器相互作用；多尺度融合；光谱成像；赤潮检测。

期刊：International Journal of Applied Earth Observation and Geoinformation

DOI：https://doi.org/10.1016/j.jag.2024.104129

设备：GaiaField

Progressive CNN-transformer alternating reconstruction network for hyperspectral image reconstruction-A case study in red tide detection

Abstract：Spectral reconstruction technology extracts rich detail information from limited spectral bands, thereby enhancing both of the image quality and the resolution capabilities. It finds application in non-destructive testing, elevating the precision and robustness of detection. Current studies primarily focus on improving the local information perception of convolutional neural networks or modeling long-distance dependencies with Transformer. However, such approaches fail to effectively integrate global– local modeling information, resulting in poor accuracy in image reconstruction. This paper introduces a Progressive CNN- Transformer Alternating Reconstruction Network (PCTARN) to alternately utilize robust convolutional attention and transpose Trans former self-attention. A Dual-Path CNN-Transformer Alternating Reconstruction Module (DPCTARM) is proposed to dynamically introduce global–local dynamic priors at various levels to facilitate extracting high- and low- frequency features. This enhancement effectively strengthens PCTARN’s capability to discern valuable signals. To verify the proposed method, a spectral dataset based on seven selected red tide algae is collected. And a peak signal-to-noise ratio (PSNR) metric of 34.58 dB is achieved, which is at least 0.44 dB higher than the methods such as MAUN and MST++. While the Params and FLOPS are reduced by over 41.9 % and 38.4 %, respectively. Since the performance of the proposed PCTARN depends not only on image quality but also on spectral fidelity, an application of spectral detection on red tide are conducted for this purpose. Four feature bands are selected from multispectral images and reconstructed into 20-band hyperspectral images by using PCTARN. Species identification and cell concentration detection are conducted based on the reconstructed images. The results demonstrate that PCTARN can enhance the spatial signal and spectral peak differences of red tide samples, achieving an identification accuracy of 94.21 % and a coefficient of determination (R2) of 0.9660 in species identification and cell concentration detection, which are respectively improved by 11.55 % and 11.59 % compared to those of 4-band multispectral detection.

Keywords：Spectral reconstruction, CNN-Transformer interaction, Multiscale fusion, Spectral imaging, Red tide detection.