Infrared and visible image fusion based on multi-scale transform and sparse low-rank representation

Yangkun Zou^1,2Jiande Wu^1,3Bo Ye^4,5*Honggui Cao^4,5Jiqi Feng⁶Zijie Wan^4,5Shaoda Yin^4,5

• ¹School of Information Science and Engineering, Yunnan University, Kunming, Yunnan Province, China
• ²Faculty of Civil Aviation and Aeronautics, Kunming University of Science and Technology, Kunming, Yunnan Province, China
• ³Yunnan Key Laboratory of Intelligent Systems and Computing, Yunnan University, Kunming, China
• ⁴Faculty of Information Engineering and Automation, Kunming University of Science and Technology, Kunming, Yunnan Province, China
• ⁵Yunnan Key Laboratory of Intelligent Control and Application, Kunming University of Science and Technology, Kunming, China
• ⁶Guangxi Huasheng new material Co., LTD, Fangchenggang, China

In order to improve the fusion of infrared and visible images, a novel and effective fusion method is proposed based on multi-scale transform and sparse low-rank representation in this paper. Visible and infrared images are first decomposed to obtain their low-pass and high-pass bands by Laplacian pyramid (LP). Second, low-pass bands are represented with some sparse and low-rank coefficients. In order to improve the computational efficiency and learn a universal dictionary, low-pass bands are separated into several image patches using a sliding window prior to sparse and low rank representation. The low-pass and high-pass bands are then fused by particular fusion rules. The max-absolute rule is used to fuse the high-pass bands, and max-L1 norm rule is utilized to fuse the low-pass bands. Finally, an inverse LP is performed to acquire the fused image. We conduct experiments on three datasets and use 13 metrics to thoroughly and impartially validate our method. The results demonstrate that the proposed fusion framework can effectively preserve the characteristics of source images, and exhibits superior stability across various image pairs and metrics.