In 1895 Rontgen obtained the first human medical image by X-ray, after which research of medical images gained momentum, laying the foundation for medical image fusion. With the development of both medical imaging technology and hardware facilities, a series of medical images with different characteristics and information were obtained, contributing to a key source of information for disease diagnosis. At present, clinical medical images mainly include Computed Tomography (CT) images, Magnetic Resonance Imaging (MRI) images, Single-Photon Emission Computed Tomography (SPECT) images, Dynamic Single-Photon Emission Computed Tomography (DSPECT) and ultrasonic images, etc. (Jodoin et al., 2015; Hansen et al., 2017; Zhang J. et al., 2017). It is necessary to fuse different modes of medical images into more informative images based on fusion algorithms, in order to provide doctors with more reliable information during clinical diagnosis (Kavitha and Chellamuthu, 2014; Zeng et al., 2014). At present, medical image fusion has been considered in many aspects, such as the localization of brain diseases, the detection of glioma, the diagnosis of AD (Alzheimer’s disease), etc. (Huang, 1996; Singh et al., 2015; Zeng et al., 2018).

Image fusion is the synthesis of images into a new image using a specific algorithm. The space-time relativity and complementarity of information in fused images can be fully used in the process of image fusion, contributing to a more comprehensive expression of the scene (Wu et al., 2005; Choi, 2006). Conventional methods of SPECT and CT fusion images mainly include component substitution and multi-resolution analysis (Amolins et al., 2007; Huang and Du, 2008; Huang and Jiang, 2012). Component substitution mainly refers to intensity-hue-saturation (IHS) transform, with the advantage of improving the spatial resolution of SPECT images (Huang, 1999; Rahmani et al., 2010). The limitation of transform invariance leads to difficulty in extracting both image contour and edge details. In order to solve this problem, contourlet transform was proposed by Da et al. (2006), Zhao et al. (2012), Xin and Deng (2013). Moreover, non-subsampled contourlet transform (NSCT) was also proposed to fully extract the directional information of SPECT images and CT images to be fused, providing better performance in image decomposition (Da et al., 2006; Wang and Zhou, 2010; Yang et al., 2016).

The Pulse Coupled Neural Network (PCNN) was discovered by Eckhorn et al. (1989) in the 1990s while studying the imaging mechanisms of the visual cortex of small mammals. No training process is required in the PCNN and useful information can be obtained from a complex background through the PCNN. Nevertheless, the PCNN has its shortcomings, such as the numerous parameters and the complicated process of setting parameters. Thus, novel algorithms to optimize the PCNN parameters has been introduced to improve the calculation speed of PCNN (Huang, 2004; Huang et al., 2004; Jiang et al., 2014; Xiang et al., 2015). SFLA is a new heuristic algorithm first presented by Eusuff and Lansey, which combines the advantages of the memetic algorithm and particle swarm optimization. The algorithm can search and analyze the optimal value in a complex space with fewer parameters and has a higher performance and robustness (Samuel and Asir Rajan, 2015; Sapkheyli et al., 2015; Kaur and Mehta, 2017).

In our study, a new fusion approach based on the SFLA and PCNN is proposed to address the limitations discussed above. Our proposed method not only innovatively uses SFLA optimization to effectively learn the PCNN parameters, but also produces high quality fused images. A series of contrasting experiments are discussed in view of image quality and objective evaluations.

The remaining part of the paper is organized as follows. Related work is introduced in Section “Related Works.” The fusion method is proposed in Section “Materials and Methods.” The experimental results are presented in Sections “Result” and “Conclusion” concludes the paper with an outlook on future work.

Image fusion involves a wide range of disciplines and can be classified under the category of information fusion, where a series of methods have been presented. A novel fusion method, for multi-scale images has been presented by Zhang X. et al. (2017) using Empirical Wavelet Transform (EWT). In the proposed method, simultaneous empirical wavelet transforms (SEWT) were used for one-dimensional and two-dimensional signals, to ensure the optimal wavelets for processed signals. A satisfying visual perception was achieved through a series of experiments and in terms of objective evaluations, it was demonstrated that the method was superior to other traditional algorithms. However, time consumption of the proposed method is high, mainly during the process of image decomposition, causing application difficulties in a real time system. Noised images should also be considered in future work where the process of generating optimal wavelets may be affected (Zeng et al., 2016b; Zhang X. et al., 2017).

Aishwarya and Thangammal (2017) also proposed a fusion method based on a supervised dictionary learning approach. During the dictionary training, in order to reduce the number of input patches, gradient information was first obtained for every patch in the training set. Second, both the information content and edge strength was measured for each gradient patch. Finally, the patches with better focus features were selected by a selection rule, to train the over complete dictionary. Additionally, in the process of fusion, the globally learned dictionary was used to achieve better visual quality. Nevertheless, high computational costs also exist in this proposed approach during the process of sparse coding and final fusion performance, which may be affected by high frequency noise (Zeng et al., 2016a; Aishwarya and Thangammal, 2017).

Moreover, an algorithm for the fusion of thermal and visual images was introduced by M Kanmani et al. in order to obtain a single comprehensive fused image. A novel method called self tuning particle swarm optimization (STPSO) was presented to calculate the optimal weights. A weighted averaging fusion rule was also used to fuse the low frequency- and high frequency coefficients, obtained through Dual Tree Discrete Wavelet Transform (DT-DWT) (Kanmani and Narasimhan, 2017; Zeng et al., 2017a). Xinxia Ji et al. proposed a new fusion algorithm based on an adaptive weighted method in combination with the idea of fuzzy theory. In the algorithm, a membership function with fuzzy logic variables were designed to achieve the transformation of different leveled coefficients by different weights. Experimental results indicated that the proposed algorithm outperformed existing algorithms in aspects of visual quality and objective measures (Ji and Zhang, 2017; Zeng et al., 2017b).

In order to verify the accuracy and preservation of the edge details in our proposed method, three sets of CT and SPECT images were fused based on our method. The results of each set were compared with four fusion methods; IHS, NSCT+FL, DWT, NSCT+PCNN. In the method of NSCT+FL, images are first decomposed by NSCT to obtain high-frequency and low-frequency coefficients, and then fusion images are obtained by taking large value high-frequency coefficients and taking average value low-frequency coefficients. In NSCT+PCNN, images are decomposed by NSCT and fused by the PCNN.

In this paper, a new fusion method for SPECT brain and CT brain images was put forward. First, NSCT was used to decompose the IHS transform of a SPECT and CT image. The fusion rules, based on the regional average energy, was then used for low-frequency coefficients and the combination of SFLA and the PCNN was used for high-frequency sub-bands. Finally, the fused image was produced by reversed NSCT and reversed IHS transform. Both subjective evaluations and objective evaluations were used to analyze the quality of the fused images. The results demonstrated that the method we put forward can retain the information of source images better and reveal more details in integration. It can be seen that the proposed method is valid and effective in achieving satisfactory fusion results, leading to a wide range of applications in practice.

The paper focuses on multi-mode medical image fusion. However, there is a negative correlation between the real-time processing speed and the effectiveness of medical image fusion. Under the premise of ensuring the quality of fusion results, how to improve the efficiency of the method should be considered in the future.

Publicly available datasets were analyzed in this study. This data can be found here: http://www.med.harvard.edu/aanlib/.

CH conceived the study. GT and CH designed the model. YC and YP analyzed the data. YL and WC wrote the draft. EN and YH interpreted the results. All authors gave critical revision and consent for this submission.

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.