Medical imaging modalities such as Positron Emission Tomography (PET), Computed Tomography (CT) and Magnetic Resonance (MR) contribute significantly in all phases of cancer management (1). PET imaging plays a fundamental role in qualitative assessment of several types of cancer (2). PET images are more often assessed visually by radiologists and clinicians (3). However, PET images traditionally provide a limited number of quantitative parameters such as the maximum, minimum and peak standardized uptake value (SUVmax, SUVmean, SUVpeak) (3, 4). These parameters are commonly used for quantifying tumor characteristics. SUVmax, for example, can be used to detect occult metastatic nodes in oral cancers (5). Additional quantitative parameters in the form of texture features have been proposed and are a current research topic in quantitative PET imaging. These have the potential to improve prognosis and diagnosis of patients with cancer (6). The past few years have seen increasingly rapid advances in the field of tumor textural analysis. Radiomics may be defined as a method of the extraction of quantitative imaging textures or features that cannot be seen by the human eye (7, 8). A considerable amount of literature has been published on the use of radiomic features. For example the utility of radiomic features as predictors of patient outcome and treatment response (9, 10).

The use of radiomic features as metrics in prognosis and diagnosis for several cancers is a promising development. However, with different imaging equipment, acquisition protocols and image processing, the variation and accuracy of radiomic features remains problematic and serves as a challenge to implementing radiomic features as biomarkers (11). There have been several investigations into the effect of different variables on the stability of PET images radiomic features. The impact of PET image reconstruction settings has been investigated (12–15). Several attempts have been made to investigate the impact of other conditions including factors such as respiratory motion (16), segmentation (17) and interpolation (18)) all of which may confound the utility of PET radiomic features. Pfaehler et al. (19) investigated the impact of different variables such as image reconstruction settings, noise, discretization method, and delineation method on the repeatability of 18F-FDG PET radiomic features. In a study which set out to determine the impact of reconstruction settings on 61 texture and features, Yan et al. (20) found that variation occurred when different reconstruction settings were applied. In their study, cluster shade, and zone percentage exhibited large variations. Features such as difference entropy, inverse difference normalized, inverse difference moment normalized, low gray level run emphasis, high gray level run emphasis, and low gray level zone emphasis were found to have high stability.

Clinical studies are complex and influenced by several factors including patient physiology and organ motion. For this reason, phantom studies can be a reasonable substitute to control for bias relative to biological variability of clinical studies. Previous research involving phantom experiments have mostly dealt with homogenous phantom images, and studies that analyse heterogeneous phantom images are limited (21–23). This study therefore set out to assess not only the effect of reconstruction settings on the stability of PET radiomic features, but also the ability of the radiomic feature to distinguish between homogeneous and heterogeneous uptake patterns. For this purpose, we designed a heterogeneous PET phantom comprising of four artificially constructed tumor inserts. The phantom was scanned and images were reconstructed with different reconstruction settings including number of ordered subsets expectation maximization (OSEM) subsets, number of iterations, use of time-of-flight (TOF) and filter cut off.

Figure 6 indicates features as categorised based on the average of the COV over all tested reconstruction settings. Forty three features were found to be stable (COV≤5%) with the application of different reconstruction settings. Such stable features included GLCM (Difference entropy, Inverse difference normalised, Inverse difference moment normalised, Second measure of information correlation and 10 other features), GLRL (Short runs emphasis, Run percentage, Run entropy and 5 other features), GLSZM (Zone size entropy and 6 other features), GLDZM (Zone distance entropy and 10 other features), NGTDM (strength, coarseness, complexity).

Figure 6 also shows 20 and 13 features with moderate (5%<COV≤10%) and poor (5%<COV≤10%) stability over all reconstruction settings, respectively. Only two features show high variation (COV>20%). Unstable features included GLRLM (Long run low grey level emphasis) and GLSZM (Large zone low grey level emphasis).

When comparing feature groups, NGTDM features have the lowest mean COV (Figure 7). GLSZM was the most sensitive feature type to the reconstruction settings.

The main purpose of this study was to assess the stability of PET radiomic features with varying reconstruction settings involving different configurations of synthetic lesions. From those features identified as stable, we determined the subset of features that can still demonstrate distinguishable and significant differences between image regions with varying radioactive heterogeneity. A phantom study was used to assess these properties and hence remove the complexities of physiologically induced confounding variables which may be introduced if the analysis is performed in vivo.

Four arrays of radioactivity filled syringes (7 in total to represent a synthetic tumour) were placed in the phantom and imaged for 80 min. Images were reconstructed with different reconstruction settings (TOF, Subsets, Iterations and FWHM Gusion filters). We extracted 78 radiomic features (GLCM, GLRLM, GLSZM, GLDZM and NGTDM), calculations were compliant with the Image Biomarker Standardization Initiative (IBSI). We calculated the COV for each feature with varying reconstruction parameters and categorized their COV values into 4 groups (stable,moderately stable, poorly stable, unstable). The results of this study indicated that different reconstruction settings have different influences on PET radiomic features. For instance, GLCM (Difference entropy, Inverse difference normalised), GLRL (Short runs emphasis, Run entropy), GLSZM (Zone size entropy), GLDZM (Zone distance entropy) were stable against all reconstruction settings, while GLRL (Long run low grey level emphasis) were unstable against most of reconstruction settings. NGTDM (Busyness) was moderately stable against subsets and unstable against filters.

The important role of TOF is measuring the variation in arrival time of the two emitted photons leading to localizing the emission point more precisely. TOF can improve the contrast and reduce the noise, and therefore a better signal to noise ratio. Interestingly in this study, the use of TOF had the lowest impact on radiomic features. The OSEM algorithm is an acceleration of the expectation maximization (EM) algorithm. However, a trade-off exists between the number of subsets and increasing noise and image quality. In addition, increasing number of iterations leads to increased noise (27). The number of iteration and subsets were found to have similar effects on all measured radiomic features. This may be due to the fact that OSEM reconstructions with n iterations and m subsets are equivalent to m iterations and n subsets and increasing either leads to an increased product of iterations (subset × iteration) which eventually leads to elevate the noise level. The largest variation of image features occurred with changing the FWHM of the gaussian filter. The role of smoothing utilizing the gaussian filter is to improve signal to noise. However, the spatial resolution will be reduced with larger FWHM which causes a more uniform intensity distribution and hence an impact on extracted texture feature variability.

This study differs from prior works in several ways such as many more features were extracted and hence reported than previous work. We also have an increased number of lesion configurations, heterogeneity activity levels and varied reconstruction parameters. As an example, (21), (22) and (23) extracted 27, 58 and 39 radiomic features respectively, while in our study, we extracted 78 radiomic features. Furthermore, Forgacs et al. utilised only 3 different numbers of iterations, 2 number of subsets and 2 FWHM Gaussian filters (21) whilst our study is based on 6 different levels of iterations, 5 levels of subsets and 8 FWHM Gaussian filter variations. Moreover, all radiomics features in this work were compliant with the Image Biomarker Standardization Initiative (IBSI). Hence, this study provides a more encompassing analysis of our knowledge of the robustness of features against different reconstruction settings whilst also exploring the utility of those features in distinguishing between heterogeneity activity distributions via Friedmans analysis.

The present findings seem to be consistent with other research which found that varying reconstruction settings has variable influence on the stability of different PET radiomic features. As an example, Gallivanone et al. assessed the impact of different reconstruction settings (i.e. filters, iterations and subsets) on different radiomic features (22). Their results found that subsets and matrix size had lowest and greatest impact on the stability of features, respectively. In our study, in comparison to Gallivanone et al., about 19, 22, 17 features (from 36 common features) had the same COVs against subsets, iterations and filter size, respectively. Our results confirm Gallivanone et al.’s finding that dissimilarity (GLCM), Short run emphasis (GLRLM), Small zone emphasis (GLSZM), strength (NGTDM) has high stability. Low gray-level run emphasis and Long run low gray-level emphasis (GLRLM), Large zone low gray-level emphasis (GLSZM) are unstable.

Doumou et al. studied the impact of image smoothing, segmentation and quantisation on the stability of 57 heterogeneity features (28). For the 38 features in common with our study, 12 features had good agreement in the effect of FWHM Gaussian filter. As an example, Inverse difference normalised (GLCM) and strength (NGTDM) were stable and small zone low emphasis and large zone low emphasis (GLSZM) were unstable against varying Gaussian filter size in both studies.

In a study by Shiri et al., 100 radiomic features were extracted from patient and phantom images with different reconstruction settings (14). Our results are consistent with their findings, in that the Short run emphasis (GLRLM), zone percentage (GLSZM), correlation and Inverse difference moment (GLCM) have small variability against subsets and FWHM filters. In Shiri at al., four different reconstruction algorithms (OSEM, OSEM+PSF, OSEM+TOF and OSEM+PSF+TOF) were included, but in our study we only included two reconstruction algorithms in order to assess the impact of TOF, specifically OSEM with and without TOF.

In another study, Forgacs et al. used inhomogeneous tumor insert (7 syringes) placed in a cylindrical phantom and imaged with different acquisition times and reconstruction settings (21). According to their strategy, reliable heterogeneity parameters must be volume independent, reproducible, and appropriate for detecting heterogeneity levels. Entropy, Correlation, Homogeneity and Contrast were found to have low variation with varying acquisition times and reconstruction settings (21). In our study, 3 out of these 4 features were found to have very low COV when varying all of the tested reconstruction settings.

Bailly et al. assessed the robustness of 15 features with matrix size, number of iterations, Gaussian post-filtering, noise and the reconstruction algorithm (29). For the 13 features in common with our own study, 38% and 54% of them showed the same COVs in number of iterations and FWHM Gaussian filter, respectively.

There are several causes for the differences between our results and previous work in this area. Firstly, the statistical methods used to analyze the results are unique. Second, the range of categorizations differ from one study to another. For instance, we categorized the features into 4 groups based on the COV values, but in the Bailly study, they used only 3 categorizations. Furthermore, other factors such as segmentation methods, bin size and default reconstruction settings may have a considerable difference between studies.

In this study, we performed further statistical analysis to determine the ability of what we have defined as stable features in distinguishing between phantom inserts with different heterogeneity. The Friedman test (non-parametric) was used for these purposes on 43 (out of 78) features. Thirty five features were excluded in this analysis due to their instability against reconstruction settings. The Friedman test was performed for each combination (shape1 vs shape2, shape1 vs shape3, shape1 vs shape4, shape2 vs shape3, shape2 vs shape4 and shape3 vs shape4) of heterogeneity configurations to determine whether or not there was a statistically significant power in each texture feature distinguishing between image regions of the varying insert configurations when using different reconstruction parameters. This study found that 15 features demonstrated statistically significant differences (p<0.05) between all regions. Therefore, these 15 features may be reasonably considered as stable and capable of discerning, with statistical significance, differences between phantom objects with varying heterogeneity. This has, to the best of the authors’ knowledge, not previously been presented before in the PET radiomics literature.

The study has some limitations. First, the impact of interpolation, segmentation and quantization have not taken into account. Whypra et al. (18) , Leijenaar et al. (13) and Lu et al. (30) have investigated the effect of these parameters. We used a fixed isotropic voxel dimension, delineation and bin size with all of reconstructed images to minimize the impact of these parameters. This was also recommended by the IBSI (31). Second, this study like others was carried out in static conditions and did not include any radio kinetic component as we aimed to report on the stability of PET radiomic features against different PET reconstruction parameters. Third, this study is concerned with the stability of radiomic features for different image reconstruction parameters. An image phantom is used for these purposes; test-retest repeatability would measure variations in phantom filling rather than variability introduced by the reconstruction alogrithm. Future work will explore if these findings are consistent across different reconstruction algorithms provided by different manufacturers. Fourth, this study did not involve clinical data. However, the phantom study informs the variabilities that may exist in a clinical context. A similar study using clinical data may be conducted using the methods used in this study. This may serve as a future work.

The purpose of this work was to determine stable PET radiomic features that do not vary with changing PET reconstruction parameters but maintain the ability to distinguish between different synthetic tumor inserts with varying heterogeneity. Our study showed that forty three (55%) features were found to be stable against reconstruction settings. Fifteen features were found to have an ability to capture heterogeneous differences between lesions. These features are: (1) stable to reconstruction parameters and (2) capable of providing statistically significant differences in the presence of different levels of phantom designed spatial heterogeneity. Further research involving clinical data using a similar approach could contribute to a deeper understanding of the clinical application and translation of radiomic features.