Abstract
Introduction:
White matter injury (WMI) is now the major disease that seriously affects the quality of life of preterm infants and causes cerebral palsy of children, which also causes periventricular leuko-malacia (PVL) in severe cases. The study aimed to develop a method based on cranial ultrasound images to evaluate the risk of WMI.
Methods:
This study proposed an ultrasound radiomics diagnostic system to predict the WMI risk. A multi-task deep learning model was used to segment white matter and predict the WMI risk simultaneously. In total, 158 preterm infants with 807 cranial ultrasound images were enrolled. WMI occurred in 32preterm infants (20.3%, 32/158).
Results:
Ultrasound radiomics diagnostic system implemented a great result with AUC of 0.845 in the testing set. Meanwhile, multi-task deep learning model preformed a promising result both in segmentation of white matter with a Dice coefficient of 0.78 and prediction of WMI risk with AUC of 0.863 in the testing cohort.
Discussion:
In this study, we presented a data-driven diagnostic system for white matter injury in preterm infants. The system combined multi-task deep learning and traditional radiomics features to achieve automatic detection of white matter regions on the one hand, and design a fusion strategy of deep learning features and manual radiomics features on the other hand to obtain stable and efficient diagnostic performance.
1. Introduction
White matter injury (WMI) is a common brain injury disorder in preterm infants, mostly occurring in preterm infants at 24–35 weeks of gestational age. WMI often leads to various degrees of neurodevelopmental delay, cognitive impairment, and even cerebral palsy in preterm infants. Part of preterm infants with severe WMI only show delayed reaction or visual abnormalities in appearance. WMI in severe cases will cause periventricular leukomalacia (PVL), which is characterized by lesions of deep white matter around the lateral ventricle. PVL is now the major disease that seriously affects the quality of life of preterm infants and causes cerebral palsy of children (1).
Related studies (2) have shown that the incidence of WMI in preterm infants is increasing year by year. It has become one of the most critical diseases in preterm infants, which can lead to long-term neurological sequelae and even death in severe cases. At present, there is no definite and effective treatment plan for this disease, and the clinical goal is mainly focused on early diagnosis and intervention. Imaging is the only diagnostic tool for cerebral white matter injury (3).
At present, imaging is often used to assess the brain injury in preterm infants. Magnetic resonance imaging (MRI), computed tomography (CT), ambulatory eleroencephalography (aEEG), and cranial ultrasound (US) are four main methods for detecting WMI in preterm infants. Among them, MRI is the preferred method. Diffusion weighted imaging (DWI) is a major breakthrough in MRI technology in recent years, which is important in the diagnosis of brain injury in preterm infants (4, 5). However, it is costly, inflexible, and cannot be performed bedside. Some studies have shown that aEEG monitoring is of great value for the early diagnosis of WMI in preterm infants and can effectively assess the status of neurological development in preterm infants (6). Cranial ultrasound has the advantage of being a bedside tool that allows safe, reliable serial imaging. It enables assessment of the evolution of injury over time, as well as brain growth and maturation (7–9).
For PVL in preterm infants, it is recommended that cranial US should be the first choice, and cranial CT should only be used as an adjunctive test due to its radiation properties (9, 10). He Xuehua and Guan Buyun et al. studied quantitative analysis of ultrasonographic gray value in premature infants with PVL (11). They performed threshold segmentation of ultrasound images of PVL in preterm infants, edge extraction of specific region of interest (ROI), quantitative analysis of white matter contained in ROI, and calculation of its average gray value as a quantitative index for evaluation of white matter density in preterm infants. Their study showed that Snake model, a medical image analysis software can provide important information in the early diagnosis and in evaluating the prognosis of PVL. The quantitative analysis of ultrasound gray value is important for the early diagnosis of PVL in preterm infants, which not only improves the accuracy of diagnosis and reduces the operator's subjective judgment error, but also has certain guiding significance for early clinical intervention and reduction of disability.
The diagnosis of WMI by cranial ultrasound imaging is still influenced by interobserver differences. Medical image processing and computer aided diagnosis are being developed to achieve objectivity (9). A semi-automatic ultrasound texture analysis method has been developed to improve the early detection of WMI in newborns (12, 13). The quantitative analysis of ultrasound gray value is important for the early diagnosis of WMI in preterm infants, which not only improves the accuracy of diagnosis and reduces the operator's subjective judgment error, but also has certain guiding significance for early clinical intervention and reduction of disability. But texture analysis method only used little information contained in the cranial ultrasound images.
For ultrasound images, the vast majority of the current clinical practice relies on experienced clinicians' diagnosis. The information of the injury area is extracted according to the intensity and boundary of the echoes. However, different doctors with different clinical experience will have different conclusions from the same cranial US images. Therefore, the standardized evaluation of WMI based on cranial US images has clinical application value. Our study aims to provide a more accurate and comprehensive method based on cranial US images for the early diagnosis and treatment of WMI in preterm infants, in order to reduce the subjectivity of analysis errors caused by human factors. Therefore, we tried to present a diagnostic system for white matter injury in preterm infants. The system combines multi-task deep learning and traditional radiomics features to achieve automatic detection of white matter regions and obtain stable and efficient diagnostic performance of WMI.
2. Materials and methods
2.1. Patients
The data used in this study was obtained from the International Peace Maternity & Child Health Hospital. Ethical approval was waived because it was a retrospective study. In this study, 158 preterm infants with cranial US images were obtained using Philips ultrasonic instrument (EPIQ5 or CX50) with convex array probe (C8-5; 5–8 MHz). The inclusion criteria were: (1) gestational age < 37 weeks, age ≤ 7 days, (2) preterm infants underwent cranial ultrasound diagnosis and had clear B-mode ultrasound images (B-US), (3) birth mother had obstetric complications, (4) preterm infants had a clear expression of intrauterine distress or a clear history of asphyxia during labor, (5) shortly after birth, preterm infants presented with persistent increased anterior fontanelle tone and neurological symptoms.
The study population consisted of 158 preterm infants with 807 cranial US images. 126 cases in the data set were normal preterm infants and 32 cases were preterm infants with WMI. We split the data set into training cohort (110 preterm infants with 566 cranial US images) and testing cohort (48 preterm infants with 241 cranial US images) at a ratio of 7:3. Figure 1 presents an overview of the patients of the study. The training cohort consists of 88 normal preterm infants (435 cranial US images) and 22 preterm infants with WMI (131 cranial US images). The testing cohort consists of 38 normal preterm infants (196 cranial US images) and 10 preterm infants with WMI (45 cranial US images).
Figure 1
Overview of the patients of the study.
2.2. Cranial ultrasound images
Ultrasonography is a technique that uses specific sound waves to measure the data of morphology and physiology of tissue structures to understand the structure and pathological processes in the human body. B-mode ultrasound (B-US) is a two-dimensional image that is displayed on the screen in grayscale in real-time. With the clinical application of ultrasonic diagnosis technology, the technology began to be used in the diagnosis of neonatal intracranial diseases in the late 1970s. Since the fontanelle is not yet closed at birth, it provides unique diagnostic conditions for clinical ultrasound detection of intracranial lesions (14). The newborn's cranial ultrasound technology is a unique diagnostic tool. The development of neonatal cranial ultrasound technology has opened up a new way for clinicians to understand intracranial lesions in vivo. Cranial ultrasound is non-invasive, painless, inexpensive, simple, easy to perform and intuitive, and has become an indispensable routine method for the diagnosis of intracranial diseases in high-risk infants.
The anterior fontanelle of coronal plane set as the standard plane of measurement. This plane clearly shows the bilateral lateral fissure, corpus callosum, and anterior horn of the lateral ventricle. In each preterm infant, 3–6 ultrasound images were obtained in this plane. In total, we obtained 807 ultrasound image samples from 158 preterm infants. All ultrasound examinations were performed by the 3 radiologists with at least 10 years of experience. And all data had the golden standard of manual segmentation by same radiologist.
2.3. Ultrasound radiomics diagnostic system
We used computer-aided diagnostic system based on B-US for the prediction of WMI in preterm infant. There are mainly four steps: ultrasound image segmentation, feature extraction, feature selection, and classification. The overview of our ultrasound radiomics diagnostic system is presented in Figure 2.
Figure 2
Overview of our ultrasound radiomics diagnostic system.
We used the golden standard segmented manually by radiologists as the input ROIs. Features were extracted from each ROI. Then, we transferred B-US ROIs to Wavelet_A, Wavelet_V, Wavelet_D, and Wavelet_H modes (15). In each mode, 70 features were extracted from each ROI. Finally, we obtained a total of 350 features in each B-US ROI, including 70 basis features and 280 wavelet features. Among them, the 70 basis features are as follows: 16 histogram features, 23 grey-level co-occurrence matrix (GLCM) based features, 13 gray-level run length matrix (GLRLM) based features, 13 gray-level size zone matrix (GLSZM) based features, and 5 neighboring gray tone difference matrix (NGTDM) based features. The details are shown in Table 1.
Table 1
| Feature Type | Feature Name | Number of characteristics |
|---|---|---|
| Basic Features | Total number of base features | 70 |
| Histogram | 1) Energy 2) Total Energy 3) Entropy 4) Minimum 5) Maximum 6) Mean 7) Median 8) Interquartile Range 9) Standard Deviation 10) Mean Absolute Deviation 11) Range 12) Root Mean Squared 13) Uniformity 14) Variance 15) Kurtosis 16) Skewness | 16 |
| GLCM | 17) Autocorrelation 18) Joint Average 19) Cluster Prominence 20) Cluster Shade 21) Cluster Tendency 22) Sum Variance 23) Difference Average 24) Correlation 25) Difference Entropy 26) Homogeneity 1 27) Homogeneity 2 28) Sum Average 29) Dissimilarity 30) Joint Energy 31) Informational Measure of Correlation 1 32) Informational Measure of Correlation 2 33) Inverse Difference Moment 34) Joint Entropy 35) Maximal Correlation Coefficient 36) Inverse Difference 37) Inverse Variance 38) Maximum Probability 39) Difference Variance | 23 |
| GLRLM | 40) Short Run Emphasis 41) Long Run Emphasis 42) Gray Level Non-Uniformity 43) Run Length Non-Uniformity 44) Run Percentage 45) Low Gray Level Run Emphasis 46) High Gray Level Run Emphasis 47) Short Run Low Gray Level Emphasis 48) Short Run High Gray Level Emphasis 49) Long Run Low Gray Level Emphasis 50) Long Run High Gray Level Emphasis 51) Gray-Level Variance 52) Run-Length Variance | 13 |
| GLSZM | 53) Small Area Emphasis 54) Large Area Emphasis 55) Gray Level Non-Uniformity 56) Size-Zone Non-Uniformity 57) Zone Percentage 58) Gray Level Variance 59) Zone Variance 60) Low Gray Level Zone Emphasis 61) High Gray Level Zone Emphasis 62) Small Area Low Gray Level Emphasis 63) Small Area High Gray Level Emphasis 64) Large Area Low Gray Level Emphasis 65) Large Area High Gray Level Emphasis | 13 |
| NGTDM | 66) Coarseness 67) Contrast 68) Busyness 69) Complexity 70) Strength | 5 |
| Wavelet Characteristics | Wavelet transform was performed on the original B-US image to obtain wavelet_A, wavelet_H, wavelet_V and wavelet_D modes, and then the basic features in the transformed modes were extracted. | 280 |
| Total number of features extracted from cranial US images | 350 |
350 features extracted from B-US.
We used sparse representation-based classifier (SRC) for feature selection, which was proposed by Li (16). The main idea of SRC is to identify the most relevant features based on the importance index generated in each bootstrap iteration (17). The SRC is based on the assumption that all signals can be represented by a linear combination of atoms in the dictionary. The atoms used to represent the signals are equivalent to the significant features. And the features interact with each other during the representation of the signals. The coefficients of the representation correspond to the importance of the features. All features can be ranked according to their coefficients. So, the top 20% of the atoms with a prominent role are retained as significant features.
In this study, support vector machine (SVM) based cost-support vector classifier (C-SVC) was used to predict the WMI risk by selected features. SVM is a binary classification model and also a linear classifier defined on a certain feature space. By using kernel functions, SVM can be transformed into a nonlinear classifier (17). The purpose of SVM is to split the sample by finding a hyperplane. C-SVC using kernel functions has been widely used in machine learning for high-dimensional complex feature spaces especially when the samples are not balanced. In the process of searching the optimal hyperplane in high-dimensional space, C-SVC has good discriminative power and stability (18).
2.4. Deep learning modeling
Deep learning models have been widely used in clinical medicine, and it is witnessing increasing innovations in the fields of AI-aided image analysis, AI-aided lesion determination, AI-assisted healthcare management, and so on (19, 20). In this study, we used two deep learning models to implement segmentation of white matter and prediction of WMI risk of preterm infants.
2.4.1. Segmentation deep learning model
In order to find the most suitable segmentation deep learning network (SDL-Net), we compared FCN (21), U-Net (22), TransUNet (23) and Swin-Unet (24) for white matter segmentations. Based on the result of this ablation experiment (detailed in Supplementary Material), we used U-Net as our SDL-Net. U-Net is an improved segmentation network based on FCN (21) that uses only a small amount of data to train an end-to-end (image in, image out) network and works well for medical images. U-Net network includes a down-sampling path for capturing semantic information, an up-sampling path for accurate segmentation localization, and a transverse connected path. The structure is shown in Figure 3. The down-sampling path is used to extract features of the image at different scales. The up-sampling path is symmetric to the down-sampling path, and they form a U-shaped structure. The transverse connected path allows the network to propagate information of the original image directly to the higher level of high-resolution layer. This structure makes the output layers of the whole network with no missing context information comparing with the original images, and allows U-Net to be used for segmentation of small datasets and reduce the memory limitation of the graphics processing unit (GPU).
Figure 3
Structure of SDL-Net.
2.4.2. Multi-task deep learning model
In order to find the most suitable multi-task deep learning network (MTDL-Net), we compared Fast R-CNN (25), Faster R-CNN (26), Mask R-CNN (27) and RS-Net (28) for white matter segmentations and WMI status prediction. Based on the result of this ablation experiment (detailed in Supplementary Material), we used Mask R-CNN as our MTDL-Net. Mask R-CNN is an improved convolutional neural network proposed based on Faster R-CNN (25). It can accomplish target detection and semantic segmentation simultaneously. The structure of our multi-task deep learning model is divided into three main parts, which is shown in Figure 4. The first part is feature pyramid networks (FPN) (29), which is used for feature map extraction from the whole image. The second part is region proposal network (RPN) (26), which is used for ROI proposal generation. The third part is output layer for classification, bounding box (B-Box) detection, and ROI mask segmentation. There is an ROI Align layer between the second and third parts for obtaining the feature maps corresponding to each ROI. Our multi-task deep learning model has three final outputs: classification labels, B-Box, and semantic segmentation ROIs.
Figure 4
Structure of our multi-task deep learning model.
2.5. Statistical analysis
The AUC was used as an index for the evaluation of the diagnostic performance. The sensitivity (SENS), specificity (SPEC), accuracy (ACC), and positive and negative diagnostic likelihood ratios (LR+, LR−) were calculated. We adopted Dice coefficient (Dice) and Intersection over Union (IoU) to evaluate the performance of segmentation. The definitions of these indexes are detailed in Supplementary Material.
The statistical analyses were performed using MedCalc software (version 20.027; MedCalc Software bvba, Mariakerke, Belgium), SPSS software (v. 22.0; IBM Corp., Armonk, NY, USA), Python 3.7, and Matlab R2020b.
2.6. Implementation details
While training, we implemented an asymmetric cyclic sampling strategy to optimize the unbalanced dataset and augment the training dataset. In order to improve the generalization ability considering the limited medical data, we applied oversampling strategy to augment the data which belong to preterm infants with WMI during training. Additionally, the training set was artificially augmented by random translation, rotation and flipping from the original images to prevent overfitting of the networks due to the limited training dataset. More details about data augmentation techniques are described in Supplementary Material.
In SDL-Net, the max training epoch is set to 100 with a batch size of 4 empirically. The model parameters are updated via the Adam optimizer with a learning rate of . In MTDL-Net, the max training epoch is set to 100 with a batch size of 4 empirically. The model parameters are updated via the Adam optimizer with a learning rate of .
Our network was implemented in the PyTorch platform, and MATLAB was used to realize data pre-processing. The entire training process was performed on a computer equipped with an Intel Xeon 4210 CPU with 128 GB RAM, and a Nvidia GeForce RTX 2080 Ti.
3. Results
3.1. WMI prediction based on ultrasound radiomics diagnostic system
In this part, the golden standard ROIs segmented by radiologists were used in feature extraction.
While training, each cranial US ROI was extracted 350 features based on grayscale, Wavelet_A, Wavelet_V, Wavelet_D, and Wavelet_H modes. Then 350 features were ranked by the index of SRC. Finally, SVM based C-SVC was introduced to predict WMI according to a different number of features. After the examination, we find that while using 52 features, the model performed the best with an AUC of 0.885. The change of AUC with the number of features is shown in the Figure 5.
Figure 5
The change of AUC with the number of features.
While testing, consistent with the training section, 350 features were extracted from cranial ultrasound ROIs based on 5 wavelet transformed modes and were ranked by the index of SRC. We used the top 52 features in SVM classifier. As a result, the AUC of SVM based C-SVC reached 0.845 in the testing cohort. The result is listed in Table 2. Figure 6 illustrated the ROC curves of SVM based C-SVC in the training and testing cohort.
Figure 6
ROC curves of SVM based C-SVC for WMI status prediction in the training and testing set.
3.2 White matter segmentation based on SDL-Net
In clinical trials, manual segmentation of ultrasound images by radiologists will consume a lot of time. In this part, in order to reduce the labor cost of radiologists and avoid labeling manually, we tried to use SDL-Net to implement the white matter segmentations autonomous by computer.
We used training cohort to train the parameters in SDL-Net, and then used testing cohort to evaluate the segmented results. As shown in Figure 7, we can see that the segmented result by SDL-Net is slightly different from the golden standard ROIs segmented by radiologists. The result is listed in Table 4. It can be observed that our method on testing cohort has an overall Dice of 0.73, an accuracy of 0.80, and an IoU of 0.72.
Figure 7
White matter ROIs segmented by SDL-Net.
Table 2
| Dataset | AUC | ACC | SENS | SPEC | LR+ | LR− |
|---|---|---|---|---|---|---|
| Training cohort | 0.885 | 78.27 | 82.44 | 77.24 | 3.62 | 0.23 |
| Testing cohort | 0.845 | 87.88 | 75.56 | 96.43 | 21.16 | 0.25 |
The corresponding quantitative indexes for WMI status prediction in the training and testing cohort.
Table 3
| ROI used | AUC | ACC | SENS | SPEC | LR+ | LR− |
|---|---|---|---|---|---|---|
| ROIs segmented by radiologists | 0.845 | 87.88 | 75.56 | 96.43 | 21.16 | 0.25 |
| ROIs segmented by SDL-Net | 0.819 | 84.85 | 64.44 | 95.92 | 15.79 | 0.37 |
The corresponding quantitative indexes for WMI status prediction based on different ROIs in the testing cohort.
Then we used our segmented result in ultrasound radiomics diagnostic system. The result is listed in Table 3. As shown, AUC of SVM based C-SVC reached 0.819 in the testing cohort. The ROC curves of SVM based C-SVC in testing cohort is shown in Figure 8.
Figure 8
ROC curves for WMI status prediction based on different ROIs in the testing cohort.
3.3. Multi-task learning based on MTDL-Net
Multi-task learning (MTL) spontaneously learns multiple related tasks, utilizing both generic information shared across tasks and information specific to each task (30). In this part, MTDL-Net achieved automatic segmentation of infant cranial ultrasound images and used the segmented information to achieve prediction of WMI.
We used training cohort to train the parameters in MTDL-Net, and then used testing cohort to evaluate the segmented results and prediction results. Part of the segmented results shows in Figure 9. As shown, the segmented result by MTDL-Net achieved better performance than SDL-Net. Table 4 shows the results of our methods. It can be observed that segmented result on testing cohort has an overall Dice of 0.78, an accuracy of 0.81, and an IoU of 0.82.
Figure 9
White matter ROIs segmented by MTDL-Net.
Table 4
| Model | DICE | ACC | IoU |
|---|---|---|---|
| SDL-Net | 0.73 | 0.80 | 0.72 |
| MTDL-Net | 0.78 | 0.81 | 0.82 |
The corresponding quantitative indexes for white matter segmentation based on different models in the testing cohort.
In the WMI prediction task, we sent segmented result by MTDL-Net into ultrasound radiomics diagnostic system to predict WMI as a comparison. The result is listed in Table 5. As shown, AUC of our method reached 0.863 in the testing cohort, while AUC of SVM based C-SVC was 0.829. The ROC curves of our method and SVM based C-SVC in testing cohort is shown in Figure 10.
Figure 10
ROC curves for WMI status prediction based on different classifier in the testing cohort.
Table 5
| Classifier | AUC | ACC | SENS | SPEC | LR+ | LR− |
|---|---|---|---|---|---|---|
| SVM based C-SVC | 0.829 | 80.30 | 80.00 | 85.20 | 5.41 | 0.23 |
| MTDL-Net | 0.863 | 87.88 | 68.89 | 96.94 | 22.50 | 0.32 |
The corresponding quantitative indexes for WMI status prediction based on different classifiers in the testing cohort.
Meanwhile, in order to evaluate the MTDL-Net for multi-task learning, we also implemented the 5-folds cross validation in our data-driven diagnostic strategy. The details of this experiment are described in Supplementary Material. The average DICE coefficient of white matter segmentation in 5-folds cross validation is 0.76. The average AUC of WMI status prediction in 5-folds cross validation is 0.843. 5-fold cross-validation reduced the variance in the estimate of model performance. The result of 5-fold cross-validation demonstrated that MTDL-Net for multi-task learning showed great promising both in white matter segmentation and WMI status prediction in our data-driven diagnostic strategy.
4. Discussion
White matter injury is a common brain injury disorder in preterm infants, with severe cases it will lead to periventricular leukomalacia. At present, there is no definite and effective treatment for this disease, and the clinical goal is mainly focused on early diagnosis and intervention. Imaging is the only diagnostic tool for cerebral white matter injury.
The development of cranial ultrasound technology in newborns has opened up a new way for clinicians to understand intracranial lesions in vivo. Non-invasive, painless, inexpensive, easy-to-perform, and intuitive, cranial ultrasound testing technology has so far become an indispensable routine method for the diagnosis of intracranial diseases in high-risk infants, and is widely used in the examination process of infant cerebral WMI. The need for a non-destructive and efficient method to predict WMI in infants through ultrasound images is urgent because the area of WMI is too blurred and poorly characterized.
The major novelty of this paper is that we established a data-driven diagnostic strategy to evaluate the risk of WMI based on cranial ultrasound images. In short, if we already have golden standard white matter ROIs, we can only use radiomics method to evaluate the risk of WMI. But if we don't have golden standard white matter ROIs, we can use MTDL-Net to achieve automatic segmentation of infant cranial ultrasound images and use the segmented information to evaluate the risk of WMI simultaneously. The overview of our data-driven diagnostic strategy is presented in Figure 11. More details are described in Supplementary Material.
Figure 11
Overview of our data-driven diagnostic strategy.
In order to implement our data-driven diagnostic strategy, this study attempted a set of methods for preterm infants WMI prediction, starting from the traditional ultrasound radiomics diagnostic system to predict WMI, and finally investigating deep learning methods for automatic segmentation and multi-task learning. The main findings as follows:
4.1. WMI prediction
This study implemented the prediction of WMI in preterm infants through B-mode ultrasound images based on an ultrasound radiomics diagnostic system. To utilize the information contained in B-mode ultrasound images, we attempted a high-throughput feature description system for infant cranial ultrasound images, extracting a total of 350 high-throughput features including histogram, GLCM, GLRLM, GLSZM and NGTDM from the original grayscale ultrasound image and four wavelet transformed modes respectively. The feature selection is then performed using SRC combined with 100 times bootstrap technique (31). Finally, the classification model was built using a SVM based C-SVC classifier.
The ultrasound radiomics diagnostic system built with 52 features after feature selection performed the best, with an AUC of 0.845, ACC of 0.8788, SENS of 0.7556, and SPEC of 0.9643 in the testing cohort. The experiments demonstrated that ultrasound radiomics diagnostic system for cranial WMI in preterm infants is effective and has sufficient clinical application value.
4.2. White matter segmentation
In clinical trials, manual segmentation of ultrasound images by radiologists will consume a lot of time. And different radiologists with different experience may produce quite different ROIs in the same ultrasound images (32). There is an urgent need for a consistent, nondestructive and efficient method for white matter segmentation in preterm infants and guiding the clinical diagnosis and treatment process.
U-Net was proved of great performance in many clinical scenarios such as segmentation of brain vessel status(33). TransUNet and Swin-Unet are Transformer-based network, which need much more training data rather than CNN-based network. Comparing to other state-of-the-art methods, U-Net adopted in this paper obtains the best segmentation performance under the premise of amount of our data. Models based on the Transformer framework may have better performance after expanding the sample size. Therefore, U-Net based SDL-Net was applied to achieve automatic segmentation of infant cranial brain white matter ROIs based on ultrasound images. Dice coefficient of white matter segmentation reached 0.73 and the accuracy reached 0.80 in the testing set.
In order to evaluate the performance of the segmented ROIs by SDL-Net, we utilized the segmented results as the input of ultrasound radiomics diagnostic system to predict the WMI of preterm infants. As we can see, the AUC of the classifier reached 0.819 in the testing cohort, which is only slightly lower than the experiment with the golden standard ROIs segmented by radiologists.
4.3. Multi-task learning
Most deep learning methods use useful information from historical data to help analyze future data and typically requires large amounts of labeled data for training. However, certain application (e.g., medical image analysis) cannot meet this requirement because it requires a lot of manual labor to label data. In these cases, multi-task learning is a good approach that uses useful information from other related tasks to alleviate data sparsity problem (34).
Based on the assumption that all tasks or at least some of them are relevant (30), MTL aims to share useful information contained in multiple tasks to build more accurate models which work better empirically and theoretically than learning each tasks independently.
In this part, MTDL-Net is applied to achieve automatic segmentation of infant cranial US images and used the segmented information to achieve prediction of WMI. MTDL-Net extracted the feature map from the whole image through FPN and ResNet50, selected the candidate regions through RPN, obtained the corresponding feature map of each candidate regionthrough ROI Align. Finally, the segmentation mask was output through FCN, and B-Box and classification labels were output through the fully connected layer.
In the segmentation task, MTDL-Net performed best with Dice coefficient of 0.78, which is better than SDL-Net. In the classification task, MTDL-Net performed best with AUC of 0.863, also better than the ultrasound radiomics diagnostic system. And within the ROIs segmented by MTDL-Net, ultrasound radiomics diagnostic system had equivalent performance comparing with the input of golden standard ROIs segmented by radiologists. Therefore, MTDL-Net can simultaneously achieve the ROI segmentation of white matter based on ultrasound images and prediction of WMI in preterm infants.
There were also some limitations in our study. First, in practical clinical applications, ultrasound examinations are in the form of video scanning plus preservation of static images. Therefore, if the ultrasound video data can be processed directly instead of using static images, the automaticity of the system will be improved and enhanced (35, 36). In addition, due to the limited number of WMI cases, preterm infants with PVL were not categorized together for comparison. Further studies are needed to determine the potential differences in the predictive efficacy of WMI and PVL.
In conclusion, this paper presents a diagnostic system for white matter injury in preterm infants. The system combines multi-task deep learning and traditional radiomics features to achieve automatic detection of white matter regions on the one hand, and design a fusion strategy of deep learning features and manual radiomics features on the other hand to obtain stable and efficient diagnostic performance.
Statements
Data availability statement
The data that support the findings of this study are available from the corresponding author on reasonable request.
Ethics statement
Ethical review and approval was not required for the study on human participants in accordance with the local legislation and institutional requirements. Written informed consent to participate in this study was provided by the participants’ legal guardian/next of kin.
Author contributions
Conceptualization, JY, ZQ and PC; methodology, JY, ZQ and PC; software, ZY and JZ; validation, JZ, SY, ZY, JY, ZQ and PC; formal analysis, SY, ZQ and PC; investigation, JZ and SY; resources, SY, ZQ and PC; data curation, SY; writing—original draft preparation, JZ; writing—review and editing, JZ, SY, ZY, JY., ZQ. and PC; visualization, JZ and ZY; supervision, JY, ZQ and PC; project administration, ZQ and PC; funding acquisition, PC. All authors contributed to the article and approved the submitted version.
Conflict of interest
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
Publisher’s note
All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.
Supplementary material
The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fped.2023.1144952/full#supplementary-material.
References
1.
Volpe JJ . Cerebral white matter injury of the premature infant—more common than you think. Pediatrics. (2003) 112(1):176–80. 10.1542/peds.112.1.176
2.
Reiss JD Peterson LS Nesamoney SN Chang AL Pasca AM Mari I et al Perinatal infection, inflammation, preterm birth, and brain injury: a review with proposals for future investigations. Exp Neurol. (2022) 351:113988. 10.1016/j.expneurol.2022.113988
3.
Back SA . White matter injury in the preterm infant: pathology and mechanisms. Acta Neuropathol. (2017) 134(3):331–49. 10.1007/s00401-017-1718-6
4.
Li H Chen M Wang J Illapani VSP He L . Automatic Segmentation of Diffuse White Matter Abnormality on T2-weighted Brain MR Images Using Deep Learning in Very Preterm Infants. Radiol Artif Intell. (2021) 3(3):e200166. 10.1148/ryai.2021200166
5.
van't Hooft J van der Lee JH Opmeer BC Aarnoudse-Moens CS Leenders AG Mol BW et al Predicting developmental outcomes in premature infants by term equivalent MRI: systematic review and meta-analysis. Syst Rev. (2015) 4:71. 10.1186/s13643-015-0058-7
6.
Kong AHT Lai MM Finnigan S Ware RS Colditz PB . Background EEG features and prediction of cognitive outcomes in very preterm infants: a systematic review. Early Hum Dev. (2018) 127:74–84. 10.1016/j.earlhumdev.2018.09.015
7.
Leijser LM de Brune FT Grond JVD Steggerda SJ Walther FJ Wezel-Meijler GV . Is sequential cranial ultrasound reliable for detection of white matter injury in very preterm infants?Neuroradiology. (2010) 52(5):397–406. 10.1007/s00234-010-0668-7
8.
Sarkar S Shankaran S Laptook AR Sood BG Do B Stoll BJ et al Screening Cranial Imaging at Multiple Time Points Improves Cystic Periventricular Leukomalacia Detection. Am J Perinatol. (2015) 32(10):973–9. 10.1055/s-0035-1545666
9.
Agut T Alarcon A Cabaas F Bartocci M Valverde E . Preterm white matter injury: ultrasound diagnosis and classification. Pediatr Res. (2020) 87(Suppl 1):37–49. 10.1038/s41390-020-0781-1
10.
Ghotra S Vincer M Allen VM Khan N . A population-based study of cystic white matter injury on ultrasound in very preterm infants born over two decades in Nova Scotia, Canada. J Perinatol. (2019) 39(2):269–77. 10.1038/s41372-018-0294-5
11.
Xue-hua H Bu-yun G Li-ling Z . Ultrasonic diagnosis of premature infant with periventricular leukomalacia by Snake model software. Chin Clin Med Imaging. (2016) 27(12):842–6. 10.3969/j.issn.1008-1062.2016.12.002
12.
Vansteenkiste E Huysmans B Govaert P Lequin M Philips W . Texture-based classification of Periventricular Leukomalacia in preterm ultrasound images. Curr Med Imaging. (2008) 4(2):113–24. 10.2174/157340508784356761
13.
Tenorio V Bonet-Carne E Figueras F Botet F Arranz A Amat-Roldan I et al Correlation of Quantitative Texture Analysis of Cranial Ultrasound with Later Neurobehavior in Preterm Infants. Ultrasound Med Biol. (2014) 40(9):2285–94. 10.1016/j.ultrasmedbio.2014.04.009
14.
Khwaja O Volpe JJ . Pathogenesis of cerebral white matter injury of prematurity. Arch Dis Child Fetal Neonatal Ed. (2008) 93(2):F153–61. 10.1136/adc.2006.108837
15.
Lambin P Rios-Velazquez E Leijenaar R Carvalho S Van Stiphout RG Granton P et al Radiomics: extracting more information from medical images using advanced feature analysis. Eur J Cancer. (2012) 48(4):441–6. 10.1016/j.ejca.2011.11.036
16.
Li Y Namburi P Yu Z Guan C Feng J Gu Z . Voxel Selection in fMRI Data Analysis Based on Sparse Representation. IEEE Trans Biomed Eng. (2009) 56(10):2439–51. 10.1109/TBME.2009.2025866
17.
Zhang L Zhou WD Chang PC Jing L Zhe Y Wang T et al Kernel Sparse Representation-Based Classifier. IEEE Trans Signal Process. (2012) 60(4):1684–95. 10.1109/TSP.2011.2179539
18.
Zhang CH Shao XJ Li DW . Knowledge-based support vector classification based on C-SVC. First international conference on information technology and quantitative management (2013). p. 1083–90.
19.
Liu C Jiao D Liu Z . Artificial Intelligence (AI)-aided Disease Prediction. BIO Integr. (2020) 1(3):130–6. 10.15212/bioi-2020-0017
20.
Zeng F Liang X Chen Z . New Roles for Clinicians in the Age of Artiἀcial Intelligence. BIO Integr. (2020) 1(3):113–7. 10.15212/bioi-2020-0014
21.
Long J Shelhamer E Darrell T . Fully convolutional networks for semantic segmentation. 2015 Ieee conference on computer vision and pattern recognition (cvpr) (2015). p. 3431–40.
22.
Ronneberger O Fischer P Brox T . U-Net: convolutional Networks for Biomedical Image Segmentation. Med Image Comput Comput Assist Intervent. (2015) 9351:234–41. 10.1109/JBHI.2018.2865450
23.
Chen J Lu Y Yu Q Luo X Zhou Y . TransUNet: Transformers Make Strong Encoders for Medical Image Segmentation (2021).
24.
Cao H Wang Y Chen J Jiang D Zhang X Tian Q et al Swin-Unet: Unet-like Pure Transformer for Medical Image Segmentation (2021).
25.
Girshick R . Fast R-CNN. 2015 Ieee international conference on computer vision (iccv) (2015). p. 1440–8.
26.
Ren S He K Girshick R Sun J . Faster R-CNN: towards Real-Time Object Detection with Region Proposal Networks. Adv Neural Info Process Syst 28 (Nips 2015). (2017) 39(6):1137–49. 10.1109/TPAMI.2016.2577031
27.
He K Gkioxari G Dollar P Girshick R . Mask R-CNN. IEEE Trans Pattern Anal Mach Intell. (2020) 42(2):386–97. 10.1109/TPAMI.2018.2844175
28.
van der Voort SR Incekara F Wijnenga MM Kapsas G Gahrmann R Schouten JW et al Combined molecular subtyping, grading, and segmentation of glioma using multi-task deep learning. Neuro-Oncology. (2023) 25(2):279–89. 10.1093/neuonc/noac166
29.
Lin TY Dollár P Girshick R He K Hariharan B Belongie S . Feature pyramid networks for object detection. 30th IEEE conference on computer vision and pattern recognition (cvpr 2017) (2017). p. 936–44.
30.
Zhang Y Yang Q . An overview of multi-task learning. Natl Sci Rev. (2018) 5(1):30–43. 10.1093/nsr/nwx105
31.
Wang SQ Wei JM . Feature selection based on measurement of ability to classify subproblems. Neurocomputing. (2017) 224:155–65. 10.1016/j.neucom.2016.10.062
32.
Chang HH Zhuang AH Valentino DJ Chu WC . Performance measure characterization for evaluating neuroimage segmentation algorithms. Neuroimage. (2009) 47(1):122–35. 10.1016/j.neuroimage.2009.03.068
33.
Livne M Rieger J Aydin OU Taha AA Madai VI . A U-Net Deep Learning Framework for High Performance Vessel Segmentation in Patients With Cerebrovascular Disease. Front Neurosci. (2019) 13. Article 97. 10.3389/fnins.2019.00097
34.
Zhang Y Yang Q . A Survey on Multi-Task Learning. IEEE Trans Knowledge Data Eng. (2022) 34(12):5586–609. 10.1109/TKDE.2021.3070203
35.
Taha AA Hanbury A . Metrics for evaluating 3D medical image segmentation: analysis, selection, and tool. BMC Med Imaging. (2015) 15:29. 10.1186/s12880-015-0068-x
36.
Abdollahzadeh A Belevich I Jokitalo E Sierra A Tohka J . DeepACSON automated segmentation of white matter in 3D electron microscopy. Commun Biol. (2021) 4:179. 10.1038/s42003-021-01699-w
Summary
Keywords
white matter injury, cranial ultrasound image, ultrasound radiomics diagnostic system, SDL-Net, MTDL-Net, multi-task learning
Citation
Zhu J, Yao S, Yao Z, Yu J, Qian Z and Chen P (2023) White matter injury detection based on preterm infant cranial ultrasound images. Front. Pediatr. 11:1144952. doi: 10.3389/fped.2023.1144952
Received
15 January 2023
Accepted
27 March 2023
Published
20 April 2023
Volume
11 - 2023
Edited by
Hong Ni, Children's Hospital of Soochow University, China
Reviewed by
Yuqi Zhang, Shanghai Children's Medical Center, China Ming Chen, University of Cincinnati, United States
Updates
Copyright
© 2023 Zhu, Yao, Yao, Yu, Qian and Chen.
This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
* Correspondence: Zhaoxia Qian zhaoxiaqian@163.com Ping Chen 2429859650@qq.com
These authors have contributed equally to this work and share first authorship
â€These authors have contributed equally to this work
Specialty Section: This article was submitted to Pediatric Neurology, a section of the journal Frontiers in Pediatrics
Disclaimer
All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article or claim that may be made by its manufacturer is not guaranteed or endorsed by the publisher.