BRIEF RESEARCH REPORT article

Front. Photonics, 25 February 2022

Sec. Optical Information Processing and Holography

Volume 3 - 2022 | https://doi.org/10.3389/fphot.2022.835962

Compact Computational Holographic Display

  • 1. Visual Computing Center, King Abdullah University of Science and Technology, Thuwal, Saudi Arabia

  • 2. Department of Electrical Engineering & Computer Sciences, University of California, Berkeley, Berkeley, CA, United States

Article metrics

View details

6

Citations

3,3k

Views

800

Downloads

Abstract

Holographic display is an ultimate three-dimensional (3D) display technique that can produce the wavefront of 3D objects. The dynamic holographic display usually requires a spatial light modulator (SLM) with a following 4f system to eliminate the unnecessary orders produced by the grating structure of the SLM. We present a technique that displays the images without the 4f system. We detect the unnecessary wavefield by phase-shifting holography and suppress it using computational optimization. Experimental results are presented to verify the proposed method.

1 Introduction

Holography could create a light field identical to the one generated by a real three-dimensional (3D) object. Thus, it is regarded as the ultimate display technique for 3D displays (Hong et al., 2011). One of the most challenging issues in holographic displays is modulating the complex wavefield at wavelength-scale, pixel by pixel, yielding ultra-high spatial resolution. The typical light modulation devices are spatial light modulators (SLMs) or digital micromirror devices (DMDs). The commercial SLMs or DMDs are mainly phase-only or amplitude-only, and the practical realization of a complex one is challenging. Various approaches have been reported to conduct complex field modulation in optical or computational manners (Hsueh and Sawchuk, 1978; Juday et al., 1991; Gregory et al., 1992; Neto et al., 1996; de Bougrenet de la Tocnaye and Dupont, 1997; Arrizón et al., 1998; Birch et al., 2000; Hsieh, 2007; Bingxia Wang et al., 2021). The optical approaches usually achieve complex modulation by integrating and controlling phase-only or amplitude-only SLMs in different ways (Juday et al., 1991; Gregory et al., 1992; Neto et al., 1996; de Bougrenet de la Tocnaye and Dupont, 1997; Hsieh, 2007), while the computational techniques convert the complex field into equivalent phase-only (Hsueh and Sawchuk, 1978; Birch et al., 2000) or amplitude-only (Arrizón et al., 1998; Bingxia Wang et al., 2021) ones that adapt to the physical constraints of commercial light modulators. The latter is also referred to as hologram encoding. Analytical solutions and computational optimization are two main approaches for hologram encoding. Detour phase (Arrizón et al., 1998; Bingxia Wang et al., 2021) and double-phase methods (Hsueh and Sawchuk, 1978; Song et al., 2012; Mendoza-Yero et al., 2014) are typical analytical techniques, which are fast but with compromised qualities, and can be easily applied to arbitrary complex holograms. Optimization techniques such as Gerchberg–Saxton (GS) (Gerchberg, 2002; Sun et al., 2018; Chen et al., 2020; Wu et al., 2021) are time-consuming with improved image quality. However, these are difficult to be applied to arbitrary complex fields (Chakravarthula et al., 2019). The rasterization structure of SLM or DMD also introduces multiple orders. The fill factor of the pixels results in the zero-order issue, which interferes with and distorts the modulated light wave. To improve the displayed image quality, it requires suppression of the zero-order. The 4f optical filter system (Arrizón et al., 2007; Liang et al., 2012; Ronzitti et al., 2012; Siemion et al., 2012; Improso et al., 2014) is a common approach that can block the zero-order in the Fourier plane. The 4f system and other variants of zero-order blocking techniques result in a non-accessible region in the final reconstruction because any part of the desired pattern near the zero-order area would also be affected. An ideal solution would be to create a correction beam with the same profile as the zero-order together with the desired target (Palima and Daria, 2007), which could create a destructive interference with the unwanted beam. However, developing complex field optimization may be the barrier to this approach (Improso et al., 2017).

Herein, we propose a computational holographic display technique that integrates phase-shifting holography and automatic differentiable (AD) optimization. The former helps us detect the unwanted complex wavefield in a holographic display system, and the latter achieves complex field optimization to suppress the unwanted wavefields. It shows that optimization through AD can obtain a phase-only hologram that acts equally as a complex field in an efficient way. In the simulation, the peak signal-to-noise (PSNR) and structural similarity index measure (SSIM) can reach 50 dB and 0.9 in around one second. Thanks to the powerful AD optimization, we can achieve comparable image quality in hologram display while deputing the 4f system. We present the methodology in Section 2 and experimental results in Section 3.

2 Methods

Figure 1A represents a typical holographic display system, wherein a laser light beam illuminates an SLM that shows the holograms. The diffracted light goes through a 4f system and reproduces the desired objects. The 4f system, as denoted by the dashed rectangle in Figures 1A, is used to block unwanted terms induced by the rasterized SLM and its limited fill factor. It usually keeps only the first order of the diffraction (Zaperty et al., 2018). The 4f makes the system bulky and may introduce extra aberrations if the focal lengths of the lenses are small. Therefore, it is demanded to be eliminated. The ideal display system is shown in Figure 1B.

FIGURE 1

FIGURE 1

Holographic display with (A) and without (B) 4f system.

However, without the 4f system, the detected images along the optical axis is an interference pattern between the directly reflected beam and the diffracted beam from the SLM, as shown in Figure 2 and Eq. 1:

FIGURE 2

FIGURE 2

Zero-order issue in holographic display systems with an SLM.

where and are the beams from the dead area and activate area of the SLM, respectively, is the free-space propagator of a complex amplitude of a wavefield with a distance z (described in Section 2.3), and I(x, y) is the detected image in the camera sensor. The proposed method detects the complex amplitude of the wavefield udead(x, y) and eliminates it with a computational optimization approach. It should be mentioned that udead(x, y) is not only the directly reflected beam from the SLM but also contains some system aberrations. We obtain it through phase-shifting holography. With the known udead(x, y), the computationally designed phase hologram displayed by the active area of the SLM can cancel it in the camera plane. We achieve this by automatic differentiable complex field optimization. The following two sub-sections describe the proposed method in more detail.

2.1 Unwanted Wavefront Detection

Figure 2 and Eq. 1 show the unwanted complex amplitude of the wavefield we want to suppress is an interference pattern. Obtaining it is a holographic imaging problem. Therefore, we introduce the four-step phase-shifting holography (Yamaguchi and Zhang, 1997; Jeong et al., 2008) to measure it. We treat the wavefield udead(x, y) at the camera sensor plane as the target and the wavefield from the SLM active area as the reference beam and perform phase-shifting hologram recording. The four holograms are taken by shifting the reference beams’ phase with a step of 0.5π; that is, α is of 0π, 0.5π, π and 1.5π, respectively. Thus, the captured holograms under phase-shifting α can be represented by

Thus, the phase and amplitude of the light beam of can be obtained by (Jeong et al., 2008)

It should be mentioned that we suppose the dead area of the SLM acts as a “mirror.” However, the cover glass of the SLMs may be imperfect, which could result in an inaccurate estimation of the zero-order.

2.2 Unwanted Wavefront Suppression Through Automatic Differentiable Optimization

With the detected complex amplitude of the wavefield udead(x, y), we need to calculate a desired phase only hologram that can be loaded into the active area of the SLM to produce the desired image after interfering with udead(x, y). Suppose b is the known target, the problem is to find a phase ϕ(x, y), such that for a known distance z, the interference between the propagated wavefield and udead(x, y) approaches the target b, which can be presented asWe can optimize the above equation using a gradient descent. At iteration n, update ϕn+1 given a step size τ:with the derivatives ofComputing is critical because the partial derivatives indicate how the phase affects the error metric locally. The analytic expression of Eq. 7 is typically derived by writing an explicit expression for the error metric and symbolically differentiating with respect to each of the input parameters. Herein, we can expand Eq. 7 using the chain rule as follows:wherewhere H is the transfer function of free-space wave propagation, ◦ is the element-wise product, and and are the Fourier and inverse Fourier operator, respectively. Calculating Eq. 8 is mathematically straightforward, but somewhat laborious. Herein, we apply the reverse-mode AD (Griewank and Walther, 2008), which is a cheap technique for a computing derivative of a scalar function with many variables by the chain rule (Blennow, 2018; Congli Wang et al., 2021). From Eqs 8, 9, it is clear that we need to optimize real-valued loss functions with complex variables, that is, . However, a non-constant real-valued function of a complex variable is not (complex) analytic and therefore is not differentiable. Generally, the same real-valued function viewed as a function of the real-valued real and imaginary components of the complex variable can have a (real) gradient when partial derivatives are taken with respect to those two (real) components, that is, . However, taking the real or imaginary part of a complex number (Peng et al., 2020; Chen et al., 2021), do not satisfy the Cauchy–Riemann equations and cannot be addressed via a complex differentiation. In this work, we use the Wirtinger derivative (Remmert, 1991; Kreutz-Delgado, 2009), which can rewrite a real differentiable function f(z) as two-variable holomorphic function f(z, z*), where z = x + jy and z* = x − jy. We can use the chain rule to establish a relationship between partial derivatives of and the partial derivatives with respect to the real and imaginary components of z:

From the aforementioned equations, we get the classic definition of Wirtinger calculus:

For step s and loss L, we have . This tells us that we can simplify the complex variable update formula above to only refer to the conjugate Wirtinger derivative , giving us exactly the step we take in optimization.

The complex numbers are represented by the cascading of real and imaginary parts in the last dimension. The derivative implementation of the complex elementary functions follows the gradient rules in Table 1 of Jurling and Fienup (2014). We implement the proposed method with PyTorch 1.8 (Chilamkurthy and Tanamala, 2019), while the derivatives are calculated automatically. All presented results in the following sections were performed on a workstation with an Intel Core i7-6820 CPU and an NVIDIA GTX1080 GPU. Figure 3 shows the image quality of the proposed AD and the traditional Gerchberg–Saxton (GS) concerning the computation time. It takes around one second to optimize one 1,024 × 1,024 hologram with a PSNR ratio more significant than 70 dB and SSIM larger than 0.8. In comparison, the GS method can hardly reach a PSNR larger than 20 dB and SSIM larger than 0.6.

FIGURE 3

FIGURE 3

Image quality of the proposed AD and the traditional Gerchberg–Saxton (GS) versus computation time.

2.3 Free-Space Wavefield Diffraction

The wave propagation mentioned in the previous sections is free-space diffraction and is implemented with the angular spectrum method (ASM) of the Rayleigh–Sommerfeld diffraction (Goodman, 2005):where u(x, y, z) is the complex amplitude at a plane located at z, is the Fourier transform, and H is the ASM transfer function given by

3 Experiments

The SLM in the experiment is HOLOEYE PLUTO with a full resolution of 1,920 × 1,080 pixels and 8 µm pixel pitch. Moreover, the image sensor is Point Grey Research GS3-U3-50S5C, with a resolution of 2,248 × 2,048 pixels and 3.45 µm pixel pitch. The light source was a laser diode with a center wavelength of 532 nm, and the image sensor distance was 250 mm away from the SLM. The four phases used for detection udead(x, y) are shown in the above row of Figure 4A, and the corresponding captured holograms are shown in the below row. The phase-shifting hologram reconstruction is shown in Figure 4B, and the propagated wavefield at the SLM plane is shown in Figure 4C. We conducted a calibration to match the size difference between the image sensor and the SLM.

FIGURE 4

FIGURE 4

Four phases and the holograms (A), and the calculated wavefield at the image sensor (B) and SLM (C) planes.

For a target image shown in Figure 5A, we calculated the phase-only holograms without and with the unwanted term suppression, which are shown in the left column of Figures 5B,C. Whether the difference between the conventional and the proposed method is optimizing the zero-order term or not, the computational calculation time is the same. However, we conducted the phase-shifting holography in the proposed method to obtain the zero-order field. The calculation time can be overlooked, but the capture takes time. This can be improved by automatically synchronizing and controlling the SLM and the camera. The corresponding experimentally reconstructed images are shown in the right columns. Comparing the proposed method with the conventional one, we can observe less background noise in Figure 5C, indicating that the proposed technique suppresses some of the unwanted terms. However, some noise still exists, which may be due to the inaccurate modeling of the optical system. This could be improved if we further consider the laser speckle and the SLM’s fill factor that reflects only part of the incident light wave.

FIGURE 5

FIGURE 5

Experimental results of conventional (B) and proposed (C) methods for displaying a target image (A).

4 Discussion and Conclusion

We presented a computational holographic display technology that can achieve a lightweight holographic display with high quality. The results show that the automatic differentiable complex wavefield optimization can suppress the unwanted wavefield with the assistance of phase-shifting holography. The automatic differentiable complex wavefield optimization can also be applied to other optical systems requiring aberration or system error corrections.

Statements

Data availability statement

The original contributions presented in the study are included in the article/Supplementary Material, further inquiries can be directed to the corresponding author.

Author contributions

NC proposed the idea and conducted the simulation and experiments. CW and WH worked together on the manuscript writing.

Funding

This work was supported by the KAUST individual baseline funding.

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher’s note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations or those of the publisher, the editors, and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

References

Summary

Keywords

holography, computational display, optimization, display, imaging

Citation

Chen N, Wang C and Heidrich W (2022) Compact Computational Holographic Display. Front. Photonics 3:835962. doi: 10.3389/fphot.2022.835962

Received

15 December 2021

Accepted

31 January 2022

Published

25 February 2022

Volume

3 - 2022

Edited by

Yaping Zhang, Kunming University of Science and Technology, China

Reviewed by

Jia Jia, Independent Researcher, Shenzhen, China

Tomasz Kozacki, Warsaw University of Technology, Poland

Updates

Copyright

© 2022 Chen, Wang and Heidrich.

This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

*Correspondence: Ni Chen,

This article was submitted to Optical Information Processing and Holography, a section of the journal Frontiers in Photonics

Disclaimer

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article or claim that may be made by its manufacturer is not guaranteed or endorsed by the publisher.

Outline

Figures

Cite article

Copy to clipboard


Export citation file


Share article

Share on WeChat

Scan with WeChat to share this article

Article metrics