High Dynamic Range Structured Illumination Microscope Based on Multiple Exposures

Structured illumination microscope (SIM) can double the spatial resolution by using fringed pattern illumination. However, for samples with large intra-scene dynamic ranges, such as clustered objects, SIM fails to reconstruct high-quality images and often exhibits strong artifacts. Herein, we present a high dynamic range SIM (HDR-SIM) method using a multi-exposure acquisition strategy. With HDR-SIM, individual and clustered microspheres and vesicles with 420 times intensity differences can be visualized in the same scene while the delicate structures of the sample were preserved effectively.


INTRODUCTION
SIM is a super-resolution fluorescence imaging method that can provide adequate structure information two times beyond the Diffraction-Limit [1]. Compared with other super-resolution methods [2], SIM stands out for low-photon bleaching and compatibility for most fluorescence labeling protocols. SIM's wide-field nature makes it fit for live-cell imaging at high speed. As a result, SIM is now widely applied in studying subcellular structures [3,4], organelle interactions [5], endocytic dynamics [6]. However, the limited dynamic range has prevented SIM from being used for biological samples like clustered vesicles or actin filaments.
In addition to imaging resolution and speed, the dynamic range is another essential factor to be concerned with when selecting imaging modalities for particular applications. An imaging system's dynamic range determines the darkest and brightest signals that it can detect [7]. When the dynamic range of an imaging system cannot cover the intra-scene dynamic range (IDR) of samples, a trade-off must be made between the loss of weak signal and intense signal saturation. It becomes even more challenging for SIM imaging. The large IDR can lead to spatial variations in the optimal values of SIM reconstruction parameters and eventually apparent artifacts. Notably, the side-lobe artifacts of structures with strong signals might overwhelm the delicate structures with weak signals. Several attempts were proposed to eliminate the artifacts in SIM, such as Wiener filtering [8], Least-Squares solver [9], Richardson-Lucy deconvolution [10], and Hessian deconvolution [11]. But they are not sufficient for samples with large IDR. Another approach to solve the problem is to divide the whole image into several subregions and apply different reconstruction parameters for each one, as in Tiled-SIM [12]. But it is hard to tile into suitable subregions when the strong signals and weak signals are mixed irregularly. As a result, SIM is considered to be not suitable for imaging samples with large IDR.
Several approaches were developed in the photography field for high dynamic range (HDR) imaging [13]. A commonly used method is the multiple exposure technique [14]. A set of low dynamic range (LDR) images is taken with progressively increased exposure and then mathematically fused into an HDR image. The multi-exposure method was recently introduced into confocal laser scanning microscopy and twophoton microscopy [15,16]. Another useful approach is adaptive illumination that uses a spatial light modulator to spatially control the light intensity distribution so that all points can be exposed reasonably [17,18]. Yang et al. reported an approach to extend the dynamic of multiphoton microscopy based on a nonuniform illumination pattern by real-time negative feedback to modulate the illumination [19]. But by now, there is no prior report about extending SIM's dynamic range to our knowledge.
In this study, we introduce an HDR-SIM method based on the multi-exposure strategy. The 8-bit spatial light modulator (SLM) grating patterns are used to illuminate the sample with different intensity and fringe patterns. Sets of LDR-SIM raw images are recorded and fused into a set of HDR raw images using a pixel-wise weighting algorithm. Finally, the HDR-SIM image is achieved by using a Wiener-filter-based reconstruction algorithm. With the proposed approach, single and clustered fluorescent beads with 420 times intensity differences can be distinguished. HDR-SIM was also used to study vesicles in live osteosarcoma cells, in which the individual dim vesicles and inter-structures of bright vesicle clusters can be visualized simultaneously.

PRINCIPLE OF HDR-SIM
In SIM imaging, the measured intensity D(r) of a fluorescence specimen S(r) can be expressed as where I(r) stands for the structured illumination intensity distribution, ⊗ denotes the convolution operator, PSF(r) is the point spread function. When cosine fringe patterned light fields are used to excite fluorescence, I(r) can be expressed as where I 0 is the average illumination intensity, m is the modulation depth, k 0 and ϕ is the spatial frequency and the bias phase, respectively. Generally, nine raw images are captured with three spatial frequencies rotated by 60°, and three phases shifted by 2π/3 for each spatial frequency. The average intensity I 0 is constant for traditional SIM. Here, to achieve HDR-SIM imaging, a set of SIM raw images LDR n (r) are captured with varying average illumination intensities I n .
The manipulation of illumination intensity is realized by changing the grayscale of fringe patterns for a laserinterference SIM. Similar to the HDR method used in photography field [14], the combined HDR image from n exposures can be calculated by a weighted sum: HDR(r) n ρ n (r) · LDR n (r) n ρ n (r) In which, the coefficient matrix ρ(r) is defined as a hat function: where α, β are the minimum and maximum brightness values in all images LDR n . For most cases, three exposures with average intensity ratio η (0.25 : 0.5 : 1) allow reasonable HDR-SIM reconstruction. Taking more exposure to data processing might improve HDR performance but decrease the imaging speed.
Since LDR n (r) are recorded with the same spatial frequency and the same initial phase but with different average illumination intensities, HDR(r) can also be expressed as S HDR (r) n ρ n (r) · η n n ρ n (r) Now we consider S HDR (r) as the new unknown sample structure information. Noting that Eq. 5 and Eq. 1 are expressed approximately in the same form, so the corresponding HDR-SIM image can be reconstructed by following the state-of-the-art SIM reconstruction procedures [20] as whereS HDR−SIM (K) is the HDR-SIM reconstruction result in Fourier space, OTF(k) is the optical transfer function, w is the Wiener parameter and A(k) is the anodization function [21].

HDR-SIM Setup
As is shown in Figure 1A, the excitation laser beam, coupled into a single-mode polarization-maintaining optical fiber, illuminates a ferroelectric SLM with 1,280 × 1,024 pixels (SXGA-3DM, Fourth Dimension Displays). Grating patterns are stored onboard and are then activated with defined timings. The grating diffracts the incoming laser beam into different diffraction orders. Only the ±1 diffraction components are allowed to pass through a spatial filter placed at the Fourier plane of lens L2, and finally form interference light fields at the sample plane. A liquid crystal variable retarders (LCVR-200-VIS, Meadowlark) and a quarter-wave plate are used to ensure that the incoming beams' polarizations are maintained s-polarized at the sample plane. Fluorescence emitted from the sample is collected by a 100×NA1.49 oil objective (UAPON, Olympus Life Science, Japan). Images are captured using a scientific complementary metal-oxide-semiconductor camera (ORCA-Flash4.0 V2, Hamamatsu). We use a constant camera exposure time for each image acquisition while manipulating equivalent illumination intensity to record sets of LDR-SIM raw images with multiexposure. Typically three sets of excitation intensities are used, corresponding to under-, middle-, and over-exposure, respectively, as shown in Figure 1B. The manipulation of equivalent illumination intensity is realized by operating the SLM on grayscale mode. Figure 1C shows the limitation of LDR imaging and the fusing process into an HDR image.

Sample Preparation
Carboxylate-modified beads (yellow-green-fluorescent 505/515, diameter 100 nm) are purchased from Thermo Fisher Scientific (F8803). The solution of yellow-green beads was diluted 50 times and was dispensed onto a coverslip. The coverslip was left in the air and dried in ambient conditions. As a result, some beads aggregated together with very high fluorescence intensity while some beads were separated as an individual bead with low fluorescence intensity.
For cell imaging, the U2OS osteosarcoma cell line was purchased from the Stem Cell Bank at the Chinese Academy of Sciences. The U2OS cells were cultured in McCoy's 5A (modified) medium (Thermo Fisher), supplemented with 1% penicillin G, streptomycin (Sangon), and 10% fetal bovine serum (Thermo Fisher) at 37°C in a 5% (v/v) CO 2 environment. Cells were transfected using Lipofectamine 2000 (Invitrogen), following the manufacturer's protocol. The cell vesicle was labeled by the CD63-EGFP vector constructed by inserting the homo species CD63 gene into the pEGFP-n1 vector (Clonetech). Twelve hours after transfection, the cells were detached using trypsin-EDTA, seeded onto poly-L-lysinecoated 35 mm glass-bottom dishes (Cellvis), and cultured in an incubator at 37°C with 5% CO 2 for an additional 24 h before the experiments.

Extending of the Dynamic Range by HDR-SIM
To validate HDR-SIM, we first created a large IDR sample with fluorescent beads. In the sample, some beads aggregated to form clusters that yielded intense fluorescence, while some individual beads scattered at other regions that showed weak fluorescence signals. The intensity ratio of the strong and weak signals can be estimated by r (I 1 · I 4 )/(I 2 · I 3 ), where I 1 is the peak fluorescence intensity of clustered beads in the under-exposure image, I 2 is the peak fluorescence intensity of single beads in the  Figure 2D). With low excitation intensity, clustered beads in bright regions can be distinguished, but the signals from individual beads are submerged in noise. At high-excitation intensities, individual beads have improved signal-to-noize ratio (SNR), but the clustered beads' signals were saturated. In comparison, the combined HDR image maintained all the scene details ( Figure 2E). The histograms of the LDR images and the combined HDR image are plotted in Figure 2F. For LDR images, the histogram's primary domain is either at the low-intensity range or the saturated range. The broad middle part of the histogram makes minor contributions to the image details. In comparison, the combined HDR image shows significantly improved uniformity on the histogram. To quantitative compare the contained information, we further calculated the entropy of each image [22], which is defined as where x i represents the lightness value, and p(x i ) is the normalized probability of x i . Figure 2G shows that the fused HDR image has the highest entropy value, indicating that our multiple exposures strategy and fusion algorithm can produce SIM raw data with more sample structure details. We further compared the estimated fringe pattern parameters from LDR raw data and fused HDR data using the crosscorrelation method [23]. As is shown in Figures 2H,I, both the period of the illumination pattern |k| and modulation depth m remained the same within <1% at each orientation, which suggests that the same spatial frequency and same phase shifts were maintained during image acquisition. show the corresponding zoomed-in image of a region with large intensity differences (red box). The fluorescent intensity of clustered beads on the right is far higher than that of the individual beads on the left. These two types of features are hard to be imaged appropriately at the same time. Figure 3D also suggested that there is a strong out-of-focus signal along with heavy background noise. By carefully adjusting the OTF attenuation parameters, the LDR-SIM image Figure 3E effectively mitigates the out-of-focus signal. It turns out to be much clearer than the wide-field image. However, in the clustered area, there are obvious artifacts most likely result from background noise. Similar scenes frequently appear when imaging actin or vesicles. In contrast, HDR-SIM image Figure 3F shows it is still possible to recognize useful structural information such as spots consisting of a small number of individual beads or lines consisting of beads huddled in one direction. In traditional SIM imaging, several issues, such as low SNR, symmetry-broken of SIM-PSF, and imperfect reconstruction parameters, could induce artifacts in the reconstructed image. For samples with a large dynamic range, the estimated reconstruction parameters were mostly determined by the bright region, which may not be perfect for the dim regions. The proposed HDR-SIM can effectively mitigate the reconstruction artifacts for two reasons. First, the combined HDR image contains information from all exposures ( Figure 2G), and the resulted SNR is higher than any single-shot image. Second, the fusion algorithm greatly enhances the uniformity of intensity distribution ( Figure 2F), which makes estimated reconstruction parameters optimal for the entire field of view. Figures 3G-I show the zoomed-in images of another region with more scattered individual beads (yellow boxes). The arrows point to typical individual beads. LDR-SIM images show obvious artifacts, especially in the dim region shown in Figure 3H. By contrast, HDR-SIM enhanced the weak signals and suppressed the strong signals. Thus, even in the weak signal region of the reconstructed image, no obvious artifacts were observed, and the individual beads demonstrated clear round shapes, as shown in Figure 3I. Figure 3J presents the intensity profiles along the white line on the two adjacent beads in Figures 3A-C. It demonstrates that HDR-SIM has the same super-resolution capability and performs better in reducing side-lobe artifacts than LDR-SIM. We counted the full width at half maximum of 60 individual beads, and the estimated resolution for HDR-SIM is 102 ± 5 nm.

HDR-SIM Imaging of Vesicles in Live Osteosarcoma Cells
Vesicles serve as the carrier of matters and signals in live cells [24,25], and they might exist as individual moieties or join together to form a large cluster, known as multivesicular bodies (MVBs) inside cells [26,27]. The MVBs can contain small intraluminal vesicles that would be released into the extracellular space in response to different cell signals. To image the individual vesicles and MVBs at the same scene is always a challenging problem owing to the large IDR. We applied HDR-SIM to image vesicles in live osteosarcoma cells. Figures 4A,B show the LDR-SIM image with moderate excitation intensity and the HDR-SIM image of the vesicles, respectively. In these images, MVBs show high brightness that dulls the fluorescence signals of individual ones. As shown in Figures 4C,D, the imaging quality of the LDR-SIM image and HDR-SIM image are similar for bright MVBs areas, and both exhibit high resolution and high contrast. In areas with weaker signals (Figures 4E,F), the HDR-SIM image maintains a high-SNR response. In contrast, the LDR-SIM image contains significant artifacts wherein the signals are almost submerged in noise. Importantly, we can now directly intraluminal vesicles inside MVBs, which are lost in LDR-SIM ( Figure 4G), which can be visualized by HDR-SIM ( Figure 4H). This will help a better understanding of the biochemical process and physiological function of MVBs.

CONCLUSION
In conclusion, we introduced an HDR-SIM imaging method for samples with a high dynamic range. It alleviates SIM reconstruction artifacts and enables visualization of detailed structures for both dim and bright regions at the same scene. The proposed method uses a set of grayscale periodic patterns loaded to SLM to achieve multiple excitation exposure. It can be easily implemented in any SLM-based SIM system without additional hardware costs. We believe that our HDR-SIM technique can be extensively adopted where dynamic range matters, such as studies with vesicles, mitochondria, neuron cell bodies, and their interactions in cell and molecular biology.

DATA AVAILABILITY STATEMENT
The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.

AUTHOR CONTRIBUTIONS
SL conceived the idea of this work. YL accomplished the experiment, performed the analysis of results, and wrote the manuscript. HL, JG, and XH gave the guide for analysis. ZL, JZ, and XJ prepared the biological samples. GW, LW, and CC did the simulation and analysis. All authors contributed to the discussion and manuscript revision.