温度引导的CNN-Transformer红外与可见光图像融合方法

岩宝; 张丽萍; 郝浩博; 王颂; 汪兵兵; 罗德林

doi:10.13568/j.cnki.651094.651316.2024.11.18.0002

您当前的位置：

首页 >

文章列表页 >

温度引导的CNN-Transformer红外与可见光图像融合方法

更新时间：2026-01-21

- 温度引导的CNN-Transformer红外与可见光图像融合方法
- Journal of Xinjiang University (Natural Science Edition in Chinese and English) Vol. 42, Issue 2, Pages: 246-256(2025)
- 作者机构：
  
  1. 厦门大学航空航天学院
  2. 昌吉学院航空学院
  3. 南方电网科学研究院
- 作者简介：
- 基金信息：
- DOI：10.13568/j.cnki.651094.651316.2024.11.18.0002
  CLC： TP391.41;TP183;TN219
- Published：2025
- 稿件说明：
移动端阅览
[1]岩宝,张丽萍,郝浩博,等.温度引导的CNN-Transformer红外与可见光图像融合方法[J].新疆大学学报(自然科学版中英文),2025,42(02):246-256.
[1]岩宝,张丽萍,郝浩博,等.温度引导的CNN-Transformer红外与可见光图像融合方法[J].新疆大学学报(自然科学版中英文),2025,42(02):246-256. DOI： 10.13568/j.cnki.651094.651316.2024.11.18.0002.

DOI：10.13568/j.cnki.651094.651316.2024.11.18.0002.

摘要

红外与可见光图像轻量化融合是计算机视觉领域研究的一项重要任务，融合模型可部署在边缘设备上实现实时融合．然而，基于CNN的方法仍然存在局限性，轻量化的实现使得模型牺牲了一定程度的融合性能，单调的卷积结构导致融合的泛化能力较低，在某些复杂场景仍然表现出不足．针对以上问题，提出了一种基于温度引导的CNNTransformer红外与可见光图像融合方法．首先引入像素预增强模块来增强输入图像，同时将Transformer与CNN结合作为特征提取与重建网络的结构，捕获红外与可见光图像之间的关联信息，提高模型的融合效果．在公开数据集及自建变电数据集上将提出的方法与其他11种融合方法进行对比分析，实验结果验证提出的算法显著提高了融合性能．

Abstract

Infrared and visible image lightweight fusion is a critical task in computer vision

aimed at achieving real-time fusion on edge devices. However

CNN-based methods exhibit inherent limitations. The lightweight architectures often entail a compromise in fusion performance

as their relatively simple convolutional structures lead to diminished generalization capabilities in the fused images

particularly under complex conditions. To address these issues

we propose a temperature-guided CNN-Transformer approach for infrared and visible image fusion.This method introduces a pixel pre-enhancement module to enhance input images and integrates Transformer with CNN as the feature extraction and reconstruction network

capturing associative information between infrared and visible images to improve fusion performance. Comparative analysis on public and self-constructed substation datasets against eleven fusion methods demonstrates that the proposed algorithm significantly enhances fusion quality.

关键词

Keywords

references

REN K Y,YAN T,HU Z X,et al.Image attention transformer network for indoor 3D object detection[J].Science China Technological Sciences,2024,67(7):2176-2190.

CHEN H L,SUN Q Y,LI F F,et al.Computer vision tasks for intelligent aerospace perception:An overview[J].Science China Technological Sciences,2024,67(9):2727-2748.

CHI H,LUO D L,WANG S.LMDFusion:A lightweight infrared and visible image fusion network for substation equipment based on mask and residual dense connection[J].Infrared Physics&Technology,2024,138:105218.

朱平哲．基于DCT与PSO的可见光与红外图像融合方法[J]．新疆大学学报(自然科学版)，2018,35(4):452-458.ZHU P Z.Method for visible and infrared image fusion using discrete cosine transform and particle swarm optimization[J].Journal of Xinjiang University(Natural Science Edition),2018,35(4):452-458.(in Chinese)

MA J Y,MA Y,LI C.Infrared and visible image fusion methods and applications:A survey[J].Information Fusion,2019,45:153-178.

PRABHAKAR K R,SRIKAR V S,BABU R V.Deep Fuse:A deep unsupervised approach for exposure fusion with extreme exposure image pairs[C]//2017 IEEE International Conference on Computer Vision(ICCV).Venice,Italy.IEEE,2017:4724-4732.

LI H,WU X J.Dense Fuse:A fusion approach to infrared and visible images[J].IEEE Transactions on Image Processing,2018,28(5):2614-2623.

ZHAO L J,YANG R L,YAN B,et al.DGFusion:An effective dynamic generalizable network for infrared and visible image fusion[J].Infrared Physics&Technology,2024,141:105495.

MA J Y,YU W,LIANG P W,et al.Fusion GAN:A generative adversarial network for infrared and visible image fusion[J].Information Fusion,2019,48:11-26.

LI Q H,YAN B,LUO D L.Infrared and visible image fusion method based on hierarchical attention mechanism[J].Journal of Electronic Imaging,2024,33(2):023011.

LIU X Z,GAO H J,MIAO Q G,et al.MFST:Multi-modal feature self-adaptive transformer for infrared and visible image fusion[J].Remote Sensing,2022,14(13):3233.

LI H,WU X J,DURRANI T S.Infrared and visible image fusion with ResNet and zero-phase component analysis[J].Infrared Physics&Technology,2019,102:103039.

GAO L,LUO D L,WANG S.Multiscale feature learning and attention mechanism for infrared and visible image fusion[J].Science China Technological Sciences,2024,67(2):408-422.

YAN B,ZHAO L J,MIAO K H,et al.TGLFusion:A temperature-guided lightweight fusion method for infrared and visible images[J].Sensors,2024,24(6):1735.

ZHANG H,XU H,XIAO Y,et al.Rethinking the image fusion:A fast unified image fusion network based on proportional maintenance of gradient and intensity[C]//The Thirty-Fourth AAAI Conference on Artificial Intelligence(AAAI-20).San Francisco,CA,USA.AAAI,2020,34(7):12797-12804.

MA J Y,CHEN C,LI C,et al.Infrared and visible image fusion via gradient transfer and total variation minimization[J].Information Fusion,2016,31:100-109.

LI H,WU X J,KITTLER J.MDLatLRR:A novel decomposition method for infrared and visible image fusion[J].IEEE Transactions on Image Processing,2020,29(2):4733-4746.

TANG L F,YUAN J T,MA J Y.Image fusion in the loop of high-level vision tasks:A semantic-aware real-time infrared and visible image fusion network[J].Information Fusion,2022,82:28-42.

MA J Y,XU H,JIANG J J,et al.DDc GAN:A dual-discriminator conditional generative adversarial network for multi-resolution image fusion[J].IEEE Transactions on Image Processing,2020,29:4980-4995.

MA J Y,ZHANG H,SHAO Z F,et al.GANMcC:A generative adversarial network with multiclassification constraints for infrared and visible image fusion[J].IEEE Transactions on Instrumentation and Measurement,2021,70:1-14.

XIONG Z,ZHANG X H,HAN H W,et al.ResCCFusion:Infrared and visible image fusion network based on ResCC module and spatial criss-cross attention models[J].Infrared Physics&Technology,2024,136:104962.

Views

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

面向水下环境的海洋生物轻量化目标检测模型

遗传算法在瞬变电磁深度学习反演中的优化策略

皮肤癌病变组织分割的多通道并行U型网络

基于锯齿空洞残差卷积的单幅图像超分辨率重建研究

基于两重网格的深度学习方法求解定常偏微分方程

Related Author

周铁军

王博

田生伟

王哲

吴文宇

张莹莹

吴新宇

谢斌

Related Institution

新疆大学软件学院

国家计算机网络与信息安全管理中心新疆分中心

新疆大学地质与矿业工程学院

新疆大学网络中心

新疆维吾尔自治区人民医院

⁰