Non-rigid Multi-Modal Medical Image Registration Based on Improved Maximum Mutual Information PV Image Interpolation Method

With the continuous improvement of medical imaging equipment, CT, MRI and PET images can obtain accurate anatomical information of the same patient site. However, due to the fuzziness of medical image physiological evaluation and the unhealthy understanding of objects, the registration effect of many methods is not ideal. Therefore, based on the medical image registration model of Partial Volume (PV) image interpolation method and rigid medical image registration method, this paper established the non-rigid registration model of maximum mutual information Novel Partial Volume (NPV) image interpolation method. The proposed NPV interpolation method uses the Davidon-Fletcher-Powell algorithm (DFP) algorithm optimization method to solve the transformation parameter matrix and realize the accurate transformation of the floating image. In addition, the cubic B-spline is used as the kernel function to improve the image interpolation, which effectively improves the accuracy of the registration image. Finally, the proposed NPV method is compared with the PV interpolation method through the human brain CT-MRI-PET image to obtain a clear CT-MRI-PET image. The results show that the proposed NPV method has higher accuracy, better robustness, and easier realization. The model should also have guiding significance in face recognition and fingerprint recognition.


INTRODUCTION
Using artificial intelligence technology to collate and analyze medical images (1)(2)(3)(4) can improve medical and health services (5) and alleviate the imbalance of medical and health resource (6)(7)(8)(9) allocation, which is the focus of the development of medical artificial intelligence. Many previous studies have mentioned that the application of artificial intelligence in the field of smart medicine should be deepened, new models and means of artificial intelligence diagnosis and treatment (10) should be promoted, and a fast and accurate intelligent medical system (11) should be established. As the main component of smart medicine, medical image plays an increasingly important role in assisting medical diagnosis and promoting the rationalization of medical decision-making. Common medical images include X-ray, computed tomography (CT) (12), magnetic resonance imaging (MRI) (13,14), positron emission tomography (PET) (15) and single-photon emission computed tomography (SPECT) (15). Medical image registration (16)(17)(18)(19) is a basic task in the process of medical image processing. In clinical practice, it is usually necessary to perform multiple imaging in multiple modes or in the same mode for the same patient, that is, to obtain information from several images and conduct comprehensive analysis. The image registration technology (20)(21)(22)(23) is used to fuse the above images, and the multi-faceted information of the human body is expressed on the same image at the same time. The internal structure and functional state of the human body are reflected from the medical image (24)(25)(26)(27), and the anatomical and physiological pathological information of the human body is more directly provided, thus playing the role of multi-information visualization at the same time.
After years of development, computer in-depth learning technology (28) and artificial intelligence technology (29) have made significant breakthroughs in theory and practice. They have made revolutionary progress in realizing the open sharing of medical information and using artificial intelligence to organize and analyze fragmented medical information. The combination of image fusion analysis and radiology is of great significance for understanding complex diseases and realizing accurate judgment, and is of great help for proposing the best diagnosis results and treatment schemes. In the process of imaging diagnosis (30,31), single image data often cannot fully display the pathological structure of patients, and pathological features need to be expressed by multi-directional image features. Compared with a single sensor, multi-sensor (32,33) can provide more extensive data for better identification of tissue or lesion details.
Some scholars focus on the improvement of fusion methods, such as Xiao and Pedrycz (34) and Xiao (35) proposed fuzzy value decision theory and extended it to the complex domain. Other scholars focus on improving interpolation algorithms, such as in 2005, Xiao (36) proposed a qualitative perturbation PV interpolation algorithm to avoid local extremum on grid points and non-grid points. In 2010, Xiao (37) proposed an improved Blackman-Harris PV (BHPV) interpolation algorithm, and introduced the Blackman-Harris window sinc function to replace the kernel function of the traditional PV interpolation. He can improve the extreme value problem of the traditional method. In 2021, Zang et al. (38) proposed an improved PV interpolation method to calculate the chord function. However, these methods do not pay much attention to the characteristics of the model itself. Therefore, how to fuse multiple image information with minimum complexity cost to obtain high reliability registration results is the primary problem to be solved in medical image registration.
To solve this problem, through the analysis and research on rigid medical image registration based on the maximum mutual information PV (Partial Volume) (39) image interpolation method, this paper proposes a decision fusion model for multiple information image data, namely the non-rigid registration model based on the maximum mutual information NPV (Novel Partial Volume) image interpolation. The cubic B-spline interpolation method can obtain better decision results under various uncertain information conditions. In addition, the effect is particularly significant in image-assisted diagnosis and treatment.
The main contributions of this paper are listed below: • The proposed NPV method combines cubic spline with DFP algorithm for the first time, and introduces image registration to make the registration result more reasonable and reliable. • By comparing the synthetic image with three original images to be registered, the image registration method proposed in this paper has great advantages in comprehensively displaying pathological details, balancing the advantages of each image, and matching the corresponding spatial points. • The improved method proposed in this paper is the traditional maximum mutual information PV image interpolation method based on gray information, so that the registration model does not need to segment the image. It is simple, accurate and robust.
The organizational structure of this paper is as follows. The second chapter focuses on the problems to be solved and the relevant implementation steps of the NPV interpolation method.
In the third chapter, some examples are introduced to prove the effectiveness of the proposed method. In the fourth chapter, the full text is summarized.

PRELIMINARIES PV Image Interpolation
PV image interpolation (40) is a common registration method in medical research, which is convenient in updating the joint histogram of two images. Suppose that the reference image is R, and the image to be registered is F for any point p on image F. After transforming T, the point q in the reference image R is corresponding, that is, q = T(p). The adjacent pixels of q are n 1 , n 2 , n 3 , and n 4 , respectively, as shown in Figure 1.  Definition 1: Let the joint histogram be h, the remaining parameters are shown in Figure 1.
Definition 2: Let f (x) be the classical PV interpolation kernel function plotted in Figure 2, defined as: In PV interpolation, the value of each gray pair in the joint histogram is the sum of several decimals with small changes, so it not only makes the calculated mutual information value more accurate, but also smooths the change of the mutual information curve when the registration has a small transformation. The process of medical image registration mainly includes four key parts, graphic transformation, registration optimization method, deformation image interpolation method and similarity measure analysis. The flow chart is shown in Figure 3. First of all, it is necessary to find the appropriate graphic transformation model and select the appropriate optimization method to solve the equation. Then, the floating image is transformed on the results obtained by solving, and the deformed image is interpolated. Finally, the similarity measure of the two images is calculated, and whether the next iteration is needed according to the similarity degree.

PROPOSED METHODS
The registration process based on image intensity is actually an optimization problem, that is, after selecting the appropriate similarity measure for the image to be registered, the similarity measure between images can be maximized as far as possible. The registration process is mainly composed of five parts, image preprocessing, image transformation, gray interpolation, objective function optimization, and similarity measure calculation. When achieving high-precision alignment of medical images, we select the reference image CT as the reference image, and select PET and MRI as floating image information sources to provide information.
In this chapter, we will focus on the five steps of the proposed method, as shown in Figure 4. First of all, we need to extract the parameter values of the graphics. The second step of image transformation is necessary in the coarse registration. In the third step, the DFP optimization algorithm is used to optimize the image. In the fourth step, the deformed graphics are interpolated. Finally, the similarity analysis of the graphics is completed and the results are output.

Preliminaries
Coordinate origin is defined in the gray center of the image, the coordinates are (c x , c y , c z ), g(x, y, z) is the gray value of point (c x , c y , c z ).
For the images to be registered, in order to improve the registration accuracy, it is necessary to remove the interference such as mask and bed frame in the image. In order to improve the registration speed and balance the relationship between the registration speed and accuracy, some voxel points are usually selected as the sample set of the objective function, namely, the images to be registered are sampled. In this paper, regular grid spacing is used for sampling. Several individual pixels are selected from CT, MRI, and PET images to be registered as sample sets.

Image Transformation
The key to register multiple different modal images is to find a suitable transformation model so that all pixels on the floating image can take you through the transformation model to find the unique pixel coordinates corresponding to them in the reference image and maintain the same or similar anatomical information.
In this paper, rigid transformation is used for global coarse registration and then non-rigid transformation is used for local elastic registration.
(1) Rigid transformation based on affine transformation Rigid transformation mainly includes linear transformation and affine transformation. The affine transformation has the ability to map the floating image line to the line in the reference image space and ensure the parallel relationship between different lines, and it can be expressed as a combination of linear transformations such as rotation, scaling and translation. Therefore, this paper uses affine transformation method for coarse registration of CT, MRI, and PET images.
Assuming that the pixel point on the original image is the spatial coordinate p 0 (x 0 ′ , y 0 ′ , z 0 ′ ) after p 0 (x 0 , y 0 , z 0 ) translation transformation, the affine transformation can be expressed as: (2) Non-rigid transformation based on cubic B-spline For a three-dimensional floating image = {(x, y, z)|0 ≤ x ≤ X, 0 ≤ y ≤ Y, 0 ≤ z ≤ Z}, the control points φ i,j,k in the image constitute the size of the grid n x , n y , n z , and limit its movement within the range of four grid points in its neighborhood, as below: Among them, λ x , λ y , λ z represents the distance of grid points on the x, y, z axis, and selects different sizes according to different registration requirements. Usually, the number that can divide the image size X, Y, Z is selected to reduce the computational complexity and ensure the stability of image transformation. The single transformation T is used to transform the spatial coordinates of the control point to ( x, y, z), and the coordinate transformation of all points in the neighborhood of the control point (x, y, z) is fitted with the change of the control point, which is expressed as (x + x, y + y, z + z). The mathematical expression of the transformation is as follows.
T(x, y, z) = Among them, i, j, k is the index of the control point coordinates ( x, y, z) in the grid, u, v, w is the distance between the corresponding point and the integer coordinate point, l, m, n is the number of B-spline basis functions, B l , B m , B n represents l, m, n basis functions, respectively. The calculation formulas of each parameter are as follows: where ⌊x⌋ , y , ⌊z⌋ represents an integer not greater than x, y, z.

DFP Optimization Algorithm
The transformation matrix is essentially an optimization problem for solving multi-parameter objective function. The use of DFP in the registration process can make the calibrated image more optimized.
In the registration process, the cubic B-spline transformation function is used as the objective function, and the transformation parameters are solved by DFP algorithm. The specific steps are as follows: (1) Given parameters δ ∈ (0, 1), σ ∈ (0, 0.5), initial point x 0 ǫR n , termination error 0 ≤ ε ≤ 1. Initially symmetric positive definite matrix H 0 and let k = 0.
(2) Calculate g k = ∇f (x k ), when g k ≤ ε stops, output x k as the approximate minimum point.
Let α k = δ mk , x k+1 = x k + α k d k , then calculate s k , y k ;

Image Interpolation Algorithm After Deformation
Since the reference image and floating image are discrete image data, the pixel coordinates are integers, and the position of the pixel coordinates of the image obtained by graphic transformation is not necessarily exactly located in the integer coordinates. Therefore, it is inevitable to interpolate the pixels on the transformed image. Interpolation belongs to a resampling method of the image. It not only needs to consider the image accuracy after interpolation, but also needs to consider the computational complexity and the protection of the original image data. PV and trilinear interpolation methods are the most widely used in interpolation calculation based on mutual information registration. In the three-dimensional case, the objective function T = f (x) satisfies the following conditions for the point m(x i , x j , x k ) of floating image MRI and the point Because cubic spline satisfies the above two conditions at the same time, using cubic spline as kernel function f (x) as below, plotted in Figure 5.
And the gray contribution histogram can be obtained as follows: Frontiers in Public Health | www.frontiersin.org  where p, q, r ∈ (0, 1) as an integer.
With the increase of kernel function order, the local extremum generated by interpolation is eliminated. However, the computational complexity will be more when the order is >3. Therefore, considering the registration time benefit and accuracy, cubic spline is selected for gray histogram calculation.
The kernel function obtained by cubic spline interpolation method is shown in Figure 6. Compared with the basic image interpolation method (Figure 2), the image interpolation curve obtained by the improved method is smoother, which effectively avoids the occurrence of burrs and sharp points. Especially, the proposed NPV method has better accuracy for changing smaller points.

Similarity Measure Analysis
Similarity measure is the consideration of calculating the similarity between reference image and floating image. This paper uses the similarity measure based on gray level, which includes the sum of squares of error, mean square error, mutual information and other methods. Since mutual information is an automatic calculation method, in order to improve the registration speed, this paper uses mutual information method to analyze the similarity measure.
The calculation of mutual information is based on Shannon entropy, which is defined as follows.
where p i represents the probability of information appearing. The mutual information between reference image CT and floating image MRI is as follows.
The joint entropy of two images is defined as follows.
He Non-rigid Multi-Modal Medical Image Registration  is used to calculate mutual information, and the registration is completed when mutual information reaches the maximum.

EXAMPLES
The brain CT, MRI, and PET images with AD were extracted from ANDI and OASIS databases for analysis. And we used Python to analyze and extract the data. Figure 7A is the CT reference image, Figure 7B is MRI reference data, and Figure 7C is PET reference data images. Medical image preprocessing includes image gray normalization, removing interference background and extracting region of interest. Define a threedimensional coordinate system for two images to be registered. Taking CT as reference image R and MRI as floating image F, the gray level of body pixels on two images from different devices is normalized to 256 gray level.
First, we use CT images as reference images and MRI images as floating images for medical registration. In Figures 8A-D are the segmented registration images of CT and MRI images, respectively. In Figure 8A, the initial registration image is preprocessed. Figure 8B is the medical image after multimodal registration. Figure 8C is a medical image of affine transformation global coarse registration. Figure 8D shows the medical image after high-precision registration of nonrigid transformation. Secondly, we use CT-MRI images as reference images and PET images as floating images for medical registration. In Figures 9A-D are partial registration images of CT-MRI and PET images, respectively. Among them, Figure 9A pre-processed initial registration image. Figure 9B is the medical image after multimodal registration. Figure 9C is a medical image of affine transformation global coarse registration. Figure 9D For the final high-precision registration results, we add false colors to facilitate physician recognition.

DISCUSSION
In one fusion, that is, when CT images and MRI images are aligned, the mutual information between the two is 0.781. In the second fusion, namely, when the CT-MRI image and PET image are aligned, the mutual information between the two is 0.622, as shown in Table 1.
It can be seen from Table 1 that the proposed NPV method has improved the mutual information of the first and second time compared with the proposed PV method, indicating that this method has higher accuracy in graph registration.
In the process of image diagnosis, it is difficult for any single image to fully display the pathological structure and case characteristics of patients. Different images have prominent advantages for pathological diagnosis, and different imaging equipment, imaging angle and image mode often lead to many differences in diagnosis results. Therefore, reasonable medical image registration is of great significance. In order to verify the new registration algorithm proposed in this paper, CT images, MRI images, and PET images are used as original images for registration. Firstly, image preprocessing (Figures 8A, 9A) and multimodal registration (Figures 8B, 9B) were performed on the above three images. Then, global coarse registration (Figures 8C, 9C), B-spline function solution, image deformation and NPV interpolation, similarity measure analysis and other steps were performed on the first two images to obtain the registration results of CT and MRI images Figure 8D). Thirdly, the registration results were re-registered with PET images, and the secondary operation was completed according to the above steps (Figures 9A-C). The final synthesis results are shown in Figure 9D.

CONCLUSION
Medical image registration is an important subject in medical research and is of great significance in medical diagnosis and medical image analysis. This paper presents a new NPV image registration model. The proposed image registration model preprocesses the image before registration, so that the subsequent registration results are more accurate and reliable. The cubic spline interpolation and DFP optimization algorithm are introduced to operate the image. In addition, the proposed method also strictly controls the similarity measure, and ensures that the similarity measure is optimal through multiple iterations. The method has strong operability. By comparing the synthetic image with three original images to be registered, the image registration method proposed in this paper has great advantages in comprehensively displaying pathological details, balancing the advantages of each image, and matching the corresponding spatial points, which plays an important role in clinical application and theoretical research. Through the specific human brain image, compared with the traditional PV image, the sensitivity test shows that the image registration model proposed in this paper has high sensitivity, high reliability and popularization to gray change. Compared with the traditional registration method, this method has higher accuracy, better robustness and is easy to implement. In the direction of face recognition and fingerprint recognition, this model should also have certain reference significance.
However, due to the different imaging principles of medical images in different modes, there may be great differences in attributes. This method is a kind of non-rigid registration, which often requires the design of feature method and the adjustment and optimization of parameters, and it is difficult to obtain a highly universal registration effect. Moreover, although the proposed NPV image registration model is easy to implement, it needs a large amount of calculation and high-performance computer software, which is of little help to the reality of remote areas. Therefore, our subsequent work is mainly to further improve the algorithm and extend it to other image recognition fields, so that it has higher applicability in areas with normal health conditions.

DATA AVAILABILITY STATEMENT
The original contributions presented in the study are included in the article/supplementary material, further inquiries can be directed to the corresponding author/s.

AUTHOR CONTRIBUTIONS
The author confirms being the sole contributor of this work and has approved it for publication.