The tissue spatial anisotropy and imperfect tissue clearing effect generally leads to heterogeneous image contrast or image blurring, limiting further image biological structure identification and analysis. We proposed a CycleGAN-based approach to restore image quality with various tissue transparency and contrast in 3D LSFM imaging. The main framework of our virtual optical clearing method is shown in Figure 1A. According to different types of tissue properties and clearing effect, we selected several representative 3D image volumes with specific structural information from the raw LSFM data to generate datasets for network training. In the tissues, 3D volumes with high image contrast, low noise and good transparency were regarded as well-cleared tissue areas. And the cropped 3D image data from the well-cleared tissue areas was performed 3D deconvolution to further suppress background signal out of focus and improve image contrast for generating the target domain image data. At the same time, 3D image volumes with severe blurring and background noise in imperfectly cleared area was selected as the source domain image data. It is noticed that unlike the supervised deep learning-based methods, we need not realize accurate data pre-alignment between the two domains in our method. Hence, various types of biological samples could be used to build datasets for our experiments, including mouse brain tissues and kidney tissues acquired by using our custom-built LSFM system.

In the specific network training process, we defined the source domain and the target domain as domain X and domain Y, respectively. Taking the GPU memory requirements of network training into account, 200 groups of unpaired 3D image volumes with 256 × 256×16 pixel³ size were randomly cropped from the domain X and domain Y to generate the training datasets. In order to improve the training efficiency and avoid overfitting, image volumes with insufficient foreground were automatically discarded before training [33]. And these image volumes were normalized and then fed into the network for training. To ensure higher generalization capability of our network, during each training process, we performed data augmentation on the input image volumes by introducing a series of random changes, such as flipping, rotation, and so on. As the loss function iteratively minimized, the non-transparent 3D tissue image identified and learned the clearer structural features from the cleared 3D tissue image. After the deep learning network was well optimized, the original large LSFM image data with varying spatial transparency and contrast was sent to the network for implementing fast image prediction and image quality restoration.

In our 3D-VoCycleGAN, we built the two generators by using ResUNet, which has been proved to have great biological feature extraction capability [34]. The ResUNet used in our generators is a kind of encoder-decoder cascade structure, which is composed of four downsampling blocks and upsampling blocks (Figure 1B). In the encoder path, each downsampling block contained a max pooling and two convolution layers, each layer of which comprised 3 × 3×3 kernel followed by a batch-normalization layer and a ReLU activation function. And the encoder encoded the input image stack into multiple feature representations at different levels through max pooling. Then the decoder consisting of four upsampling blocks symmetrical to the encoder was used to decode data information back to the original dimension. Each upsampling block consisted of trilinear interpolation, skip connection, and convolution layers. The skip connection concatenated the high-level features and spatial information between encoder and decoder, thus retaining more details and capturing finer information. Finally, we established a residual connection between the input and the output of the decoder to avoid the gradient vanishing problem and improve the performance of the network [35].

For building the discriminators, we modified the five-layer conventional structure of PatchGAN [31] into a 3D form (Figure 1C). In the PatchGAN, conventional operation with a stride of 2 and padding of 1 was used in the first three conventional layers, each layer of which was followed by an instance normalization layer, and a LeakyReLU activation function. And the channel numbers were doubled as the image resolution was halved in the first three conventional each time. And the channel numbers of the five convolutional layers were 64, 128, 256, 512 and 1, respectively. Considering that the size of the input stack was 256 × 256×16, we set the kernel size of the last two convolution layers to 3 × 3×3, making the size of the output patch 32 × 32×2. Finally, the last convolutional layer reduced the channel numbers of the feature map to 1, and a sigmoid activation function was used to normalize the output value into [0, 1].

Our 3D-VoCycleGAN was implemented based on the Pytorch deep learning framework. The learning rate of the Adam optimizer was set to 0.0002 in the first 100 epochs and linearly decayed to 0 in the next 100 epochs. The batch size was set to 1. The overall training and prediction processes based on our method were implemented in a Dell 7,920 workstation equipped with RTX 3090 GPU (24 GB memory).

The deep learning model of 3D-VoCycleGAN contained two generators G X Y (from domain X to Y), G Y X (from domain Y to X) and two discriminators D X ,   D Y . The two generators G X Y , G Y X aimed to realize transformation from 3D image volumes of domain X to 3D image volumes of domain Y and inverse image transformation from domain Y to domain X, respectively. In each domain, an image volume could be the output of the generator or come from the original training data. Hence, two discriminators were used to judge which situation the image volumes belong to. In a word, the goal of the generator was synthesizing images and fooling the discriminator, while the discriminator tried to accurately winnow truth from falsehood. The standard GAN loss [36] was given as follows: l G A N ( G X Y ) = E y ∼ p d a t a ( y ) [ log ⁡ D Y ( y ) ] + E x ∼ p d a t a ( x ) [ log ⁡ ( 1 − D Y ( G X Y ( x ) ) ] , (1) l G A N ( G Y X ) = E x ∼ p d a t a ( x ) [ log ⁡ D X ( x ) ] + E y ∼ p d a t a ( y ) [ log ⁡ ( 1 − D X ( G Y X ( y ) ) ] , (2) where x denotes the blurred stack in domain X, y denotes the clear stack in domain Y. E x ∼ p d a t a   ( x ) , E y ∼ p d a t a   ( y ) were the expectation operators. However, standard GAN loss usually suffers from mode collapse and vanishing gradients, which severely degrades network performance and stability. Here we used LS-GAN loss [37] as objective function: l G A N ( G X Y ) = 1 2 E x ∼ p d a t a ( x ) [ ( D Y ( G X Y ( x ) ) − 1 ) 2 ] , (3) l G A N ( D Y ) = 1 2 E y ∼ p d a t a ( y ) [ ( D Y ( y ) − 1 ) 2 ] + 1 2 E x ∼ p d a t a ( x ) [ ( D Y ( G X Y ( x ) ) ) 2 ] , (4) l G A N ( G Y X ) = 1 2 E y ∼ p d a t a ( y ) [ ( D X ( G Y X ( y ) ) − 1 ) 2 ] , (5) l G A N ( D X ) = 1 2 E x ∼ p d a t a ( x ) [ ( D X ( x ) − 1 ) 2 ] + 1 2 E y ∼ p d a t a ( y ) [ ( D X ( G Y X ( y ) ) ) 2 ] , (6)

The original cycle consistency loss calculated the L₁ loss between the original image and reconstructed image. We incorporated a 3D multi-scale structural similarity index metric (MS-SSIM) to construct a detail preserving transformation [38]. The cycle consistency loss was defined as follows: l c y c l e   ( G X Y , G Y X ) = E x ∼ p d a t a   ( x ) [ ‖ G Y X ( G X Y ( x ) ) − x ‖ 1 ] + E y ∼ p d a t a   ( y ) [ ‖ G X Y ( G Y X ( y ) ) − y ‖ 1 ] + η ⋅ l M S ‐ S S I M , (7) l M S ‐ S S I M ( G X Y , G Y X ) = 1 − M S ‐ S S I M ( x , G Y X ( G X Y ( x ) ) ) + 1 − M S ‐ S S I M ( y , G X Y ( G Y X ( y ) ) ) , (8) where η represents the weight factor for the l M S − S S I M term. And it was set to 0.2 in all experiments. An additional identity loss was used to constrain the generator to build an identity mapping when well-cleared volumes of the target domain were provided. l i d e n t i t y ( G X Y , G Y X ) = E x ∼ p d a t a   ( x ) [ ‖ G Y X ( x ) − x ‖ 1 ] + E y ∼ p d a t a   ( y ) [ ‖ G X Y ( y ) − y ‖ 1 ]   , (9)

Saliency constraint has been proved to be effective for content preservation on 2D microscopic images [39]. And we added it to our 3D-VoCycleGAN for maintaining fine structures and information. By appropriately setting the threshold value, we could separate the foreground with detailed structure from the 3D samples. Therefore, the two generators focused on the essential structures of foreground at the early training stage and avoided appearance of artifacts in background with the aid of saliency constraint. The saliency constraint loss can be written as:   l s a l i e n c y ( G X Y , G Y X ) = E x ∼ p d a t a   ( x ) [ ‖ T α ( x ) − T β ( G X Y ( x ) ) ‖ 1 ] + E y ∼ p d a t a   ( y ) [ ‖ T β ( y ) − T α ( G Y X ( y ) ) ‖ 1 ] , (10) where T α ( x ) = s i g m o i d [ 100 ( x − α ) ] and T β ( x ) = s i g m o i d [ 100 ( x − β ) ] are binary functions parameterized by threshold α and β.

The total loss for generators could be expressed by: L G e n e r a t o r   = l G A N ( G X Y ) + l G A N ( G Y X ) + λ ⋅ l i d e n t i t y ( G X Y , G Y X ) + ρ ⋅ l s a l i e n c y ( G X Y , G Y X ) + μ ⋅ l c y c l e   ( G X Y , G Y X ) , (11) where the parameters λ , μ , ρ denotes the weight factor for the identity loss, saliency constraint loss and cycle loss, respectively. We assigned λ = 5 , μ = 10 , respectively. And we set ρ = 10 × e − 0.1 × n , where n denotes training epoch number, making the network pay more attention to the transformation of important details and ignoring the interference from the background at the early stage of training.

The sample used in this study contained mouse brain and mouse kidney tissues. For verifying the performance of our method on tissue slices, we prepared Thy1-GFP mouse brains and mouse kidney tissues, which were sectioned into 300-μm and 200-μm thick slices, respectively. The mouse kidney tissue slices were stained with DRAQ5 for labelling the cell nucleus before cleared by FOCM reagents. The FOCM reagents were prepared as 30% (wt/vol) urea (Vetec), 20% (wt/vol) D-sorbitol (Vetec), and 5% (wt/vol) glycerol dissolved in DMSO. When preparing the reagent, urea and D-sorbitol were dissolved in DMSO and stirred at room temperature overnight. After complete dissolution, glycerol was added and stirred further. The reagents should be stored at room temperature and shaken gently before using. Before imaging experiments, the well-stained mouse brain and kidney tissue slices were incubated in FOCM reagents for several minutes.

For verifying the performance of our method on deep tissues with millimeter-thickness, we prepared stereoscopic mouse brain and kidney tissue blocks labelled with Alexa Fluor 647 anti-mouse CD31 antibody (CD31-AF647, BioLegend) by caudal vein injection. The Alexa Fluor 647 anti-mouse CD31 antibody (20 mg) was then diluted in sterile saline (total volume of 150 ml). After the injection, mice were placed in a warm cage for 30 min prior to perfusion. Then mice were rapidly anesthetized with chloral hydrate [5%, wt/vol, 0.1 ml/10 g, intraperitoneal (i.p.)] and transcardially perfused with ice-cold 0.01 M phosphate buffered saline (PBS, Coolaber) and paraformaldehyde (PFA, 4% in PBS wt/vol, Saiguo Biotechnology Co., Ltd). Mouse brains and kidneys were collected and incubated in the same PFA solution at 4°C for 24–48 h for uniform fixation. After fixation, mouse brains and kidneys were washed in 0.01 M PBS at room temperature (20–25°C) for 6–12 h. The mouse brains and kidneys were clarified by the CUBIC-L/R+ protocol [xxx]. CUBIC-L [10 wt% of N-butyldiethanolamine (Vetec) and 10 wt% of Triton X-100 (Sigma) in water] and CUBIC-R+ [45 wt% of antipyrine (Vetec) and 30 wt% of nicotinamide (Vetec) in water, buffered with 0.5% (v/w) N-butyldiethanolamine (pH ∼ 10)] was prepared for tissue clearing. Mouse organs and tissues were incubated in CUBIC-L for 7 d at 37°C with gentle shaking followed by PBS washing at room temperature. After PBS clearing, the mouse brains and kidneys was incubated and stored in CUBIC-R+ at room temperature.

For generating 3D datasets and verifying the application of our method, we acquired the experimental data of mouse organs and tissues via a custom-built LSFM system. For exciting fluorescence signal of mouse tissues labeled with Thy1-GFP, DRAQ5, and CD31-AF647, two semiconductor lasers (OBIS 488LS/637LX nm, Coherent) were aligned and expanded by a pair of achromatic lenses with 30 and 250 mm focal length, respectively. Like the classical selective plane illumination microscopy, we used a cylindrical lens with 100 mm focal length and a low-NA objective (Olympus ×4/NA 0.1/WD 18.5 mm) with long working distance to generate a thin illumination sheet. And a mechanical slit (VA100 C/M, Thorlabs) was set to 1 mm for controlling the thickness of the illumination sheet. The fluorescence signal was collected by a tube lens (ITL200, Thorlabs), detection objective (Nikon ×4/NA 0.2/WD 20 mm), a multi-channel emission filter (#87-247, Edmund Optics) and a sCMOS camera (ORCA-Flash 4.0 V3, Hamamatsu). And the biological samples were loaded on a 3D motorized stage (KMT50SE/M-3D, Thorlabs) and performed scanned imaging. In this system, we used the external trigger mode with a synchronous signal to ensure high-frame image acquisition. During the LSFM imaging process, each frame was captured with a constant acquisition interval of 2 μm and total exposure time of 10 ms. The corresponding camera acquisition speed could be up to 100 fps. Due to the rapid imaging advantage, the image acquisition process of a whole mouse brain tissue slice could be finished in 4 min. All images were transformed and stored as 16-bit Multi-TIFF format for post-processing.

Due to spatial anisotropy and structural complexity of biological tissues, chemical tissue clearing still could not realize perfect tissue transparency and image contrast. As for a pre-cleared tissue, the clearing extent and spatial image degradation is random across the whole 3D volume. Here we first demonstrated our 3D-VoCycleGAN on Thy1-GFP mouse brain slices to digitally improve the optical clearing extent and image quality. Although the whole image showed the distribution of fluorescence signal in the brain slice, some details such as nerve fiber were still blurred as shown in Figure 2A. By using our 3D-VoCycleGAN, the image background was greatly suppressed. Besides, the blurred nerve fiber in the original image became more distinguishable after enhancement. Further, we cropped a region-of-interest (ROI) from the brain slice to compare the virtual optical clearing efficiency and image enhancing performance with other deconvolution methods. Here we performed sparse deconvolution [40], Richardson-Lucy deconvolution and 3D-VoCycleGAN on a 500 × 500×32 pixel³ image volume. According to the results in Figure 2B, three methods all realized image quality improvement to different extents. Then we plotted the profiles of two lines across the nerve fiber to quantitatively evaluate their differences of performance. Although Richardson-Lucy also enhanced the image resolution to some extent, the image background noise suppression capability was not as good as the other methods. Our 3D-VoCycleGAN and sparse deconvolution showed better image contrast and SBR improvements, which were both increased by above 40% (Figure 2C). It is worth noting that although sparse deconvolution showed very powerful image quality enhancing capability comparable to our 3D-VoCycleGAN, the processing time consumption for the image volume with same size is larger than our method (Figure 2D). It usually takes a considerable amount of time to finish the image processing by using Richardson-Lucy deconvolution. And sparse deconvolution could shorten the processing time with the aid of GPU acceleration. Nevertheless, due to GPU memory limitation, sparse deconvolution could not realize image processing of large 3D data in one time. Although sparse deconvolution could process the 3D image data volume by volume after splitting the large 3D image data into several image stacks, the final stitched 3D image will show uneven brightness and background since these sub-volumes exist sparsity differences. As for this image volume, the processing time consumption of 3D-VoCycleGAN, sparse deconvolution, and Richardson-Lucy deconvolution were 0.8, 48 and 45s, respectively. Thereinto, we performed 50 iterations of the Richardson-Lucy deconvolution by using the DeconvolutionLab2 plugin in ImageJ/Fiji. Then, with 3D-VoCycleGAN, image processing of the whole brain slice with 3,100 × 3,500×180 pixel³ volume could be finished in only 141s. Hence, our 3D-VoCycleGAN could realize great image quality enhancement with short time consumption, showing the great image processing efficiency in brain slice imaging.

Tissue slice imaging with chemical optical clearing protocols are widely used in biological research. Although many chemical tissue clearing methods have promoted biological structure and function research, tissue scattering and refractive index mismatching between multiple media in the imaging system usually influence the image quality and microstructure analysis under the slice surface. Our 3D-VoCycleGAN could contribute to the information restoration of images deep inside tissue slices. As shown in Figure 3A, labelled cell nucleus of the mouse kidney tissue slice showed the distribution of glomeruli. However, the tissue transparency was not enough to distinguish the detailed nucleus or glomeruli. Especially, due to the uneven thickness and structural anisotropy in axial direction, the clearing reagents could not perfectly make the kidney tissue slice transparent. For example, although the kidney tissue slice could be optical cleared by the clearing reagents, the images deep inside the tissue slice were still blurred due to inevitable light scattering or attenuation. The fluorescence signal was nearly overwhelmed in the strong background signal except the area from the tissue surface to 115-μm depth (Figure 3B). By using our 3D-VoCycleGAN, the fluorescence signal in deep tissue could be quickly recovered. The axial images in Figures 3B,C showed the profile of glomeruli and tubules across about 150-μm depth, which improved the clearing depth of mouse kidney tissues by up to 30%. Besides, we could also compare the 2D images in different depths. When the imaging depth was 100 μm, the original image could show distinguished cell nucleus with faint noise. Our virtual optical clearing method could improve the image contrast with details maintained as shown in Figure 3D. When the imaging depth was 140 μm, the morphology of cell nucleus and glomeruli were severely blurred (Figure 3E). With 3D-VoCycleGAN, the final image quality was evidently improved, where some information of nucleus and glomeruli was recovered. Hence, to some extent, we could realize structural information restoration of imperfectly cleared tissue slices in deep depth by virtual optical clearing technique.

Except for tissue slice imaging, high-throughput imaging for large-scale 3D tissues also plays an increasingly important part in biological research, especially in digital organ mapping and brain network reconstruction. Here we demonstrated the 3D image enhancement capability for stereoscopic mouse tissue with above 1 × 1 × 1 mm³ volume by using our virtual clearing method. To further depict the glomeruli distribution or morphology of kidney tissue, we imaged and reconstructed a mouse kidney tissue block with a custom-bulit LSFM system. As shown in Figure 4A, the original 3D image volume has evident space-variant opaqueness and structure blurring. In particular, the structural contours of glomeruli were gradually blurred as the axial depth increases, which was marked by the white arrow in Figure 4A. The connections between glomeruli and arteries were also very important for supporting some kidney functions. However, in the imperfectly cleared kidney tissue, a branch of the kidney artery was also overwhelmed in the background noise (yellow arrow in Figure 4A). By using our virtual optical clearing method, the glomeruli and artery branch were all evidently recovered and distinguished (Figure 4B). And the anisotropic tissue transparency was eliminated after virtual optical clearing. From the depth color-coded z-projections, we could see the whole image contrast and fluorescence signal intensity in various depths were enhanced (Figures 4C,D).

In order to quantify the virtual optical clearing effect, we selected two 2D images from two different z-depths. As shown in Figure 4E, the original 2D image had low image quality, including strong noise, blurring, and structure missing (two blue arrows). After virtual optical clearing, the details and contours of glomeruli were enhanced greatly, which might be meaningful to the morphological analysis of kidney microstructures. Although the original 2D image in Figure 4F had lower noise and blurring extent than the original image in Figure 4E because of shallow imaging depth, the vascular walls of two artery branches were still difficult to be distinguished directly (two green arrows). Our virtual optical clearing successfully recovered the details of two vascular walls. Further, we evaluated the fluorescence signal intensity and noise level in the two depths by plotting four lines. As shown in Figure 4G, images processed by our method had more distinguishable details about glomeruli and artery, especially the contour of glomeruli and vascular walls. Besides, our method improved the SBR of images by above 25%. Meanwhile, we tested our method on 3D images of brain vessels, which showed complex structures and dense distribution in the mouse brain. For deep brain tissue 3D imaging, tissue anisotropy and imperfect tissue clearing will introduce strong scattering and attenuation of fluorescence signal, resulting in vessel blurring and artifacts as shown in Figure 4H. According to the comparison results of depth color-coded z-projections, we could see the definition and sharpness of vessels were greatly improved in various depths by using our virtual optical clearing method. As shown in Figures 4I,J, the vessel artifacts were suppressed after virtual clearing. Similar to the results of kidney tissue, the original vessel contours existed severe blurring. Our virtual optical clearing method could effectively restore the information and improve the image quality, indicating its powerful image enhancement capability in 3D fluorescence imaging.