CIT was born following the rapid development of information processing technology (Lee et al., 2022), micro-nano fabrication technology (Qian and Wang, 2010), artificial intelligence technology (Suo et al., 2021), and high-speed computing power (Ying et al., 2020), and is an innovation in photoelectric imaging technology. In a broad sense, all optical imaging methods introduced in the imaging process can be considered computational imaging. In addition, the use of the processing speed of powerful computers to assist or directly participate in the improvement of the imaging effect, such as image processing, belongs to computational imaging. In a narrow sense, CIT is driven by information, and the use of information acquisition, transmission, and interpretation to describe the optical imaging process, which is a multidisciplinary combination of new imaging technology, set optics, mathematics, and information technology.

Further, traditional optical imaging is “what you see is what you get, and what you get is what you see.” The information-centered computational imaging method combines a full-link imaging process with mathematical calculations and signal processing through information acquisition, transmission, and interpretation. In terms of the information dimension, scale, and resolution, information transmission is combined with mathematical analysis to overcome the bottleneck problems of difficult physical information acquisition, model development, and resolution in the imaging process, to achieve the imaging “get more than what you see, and get better than what you see” results.

CIT comprehensively considers the physical nature of the imaging process, promotes the movement of imaging system design from the traditional aberration-driven to information-driven methods, considers the full-link imaging process, and realizes the change in the information transmission mechanism. Its components can be divided into three aspects:

1) CIT designs imaging systems from the perspective of information transmission, improves the degrees of freedom of imaging, and fully excavates the light field information for the purpose of accurate information acquisition and transmission. This enables CIT to achieve a breakthrough in the “invisible,” “incomplete:” and other problems of traditional optical imaging technology.

2) From the perspective of the entire imaging process, CIT decomposes the traditional photoelectric imaging-independent optimization concept. The light field representation model migration to the transmission medium, and the imaging system into the imaging model driven by information transfer, give full play to the characteristics and advantages of the media optical system and information processing in the imaging link, breaking the limitations of traditional imaging.

3) CIT introduces the idea of information coding to broaden the information channel and increase the capacity of the information required for imaging in the form of active or passive coding. Through active coding methods such as light sources and optical system modulation, CIT expands the method of information acquisition and improves the efficiency of information collection. In addition, considering the encoding effect of the transmission medium on information, through the joint multiplexing of multi-dimensional physical quantities such as amplitude, phase, polarization, and spectrum, the interpretation ability of information is improved, and breakthroughs in imaging resolution, field of view, and action distance are achieved, turning “impossible” imaging into “possible.” However, CIT cannot process information that has not yet been acquired. Instead, it actively discards nonessential dimensional information and increases the amount of necessary dimensional information to improve the imaging performance and overcome the limitations of traditional imaging.

The concept of computational imaging has existed since the beginning of image processing; however, it was not until the 1990s that Athale first introduced it (Mait et al., 2012). Subsequently, Stanford University, Columbia University, the Massachusetts Institute of Technology, Boston University, and others formally initiated research on CIT. and established the Media Lab Camera Culture Group, Computational Imaging Lab, Hybrid Imaging Lab, Electrical and Computer Engineering Lab, and other laboratories that initiated research on CIT. In the USA, almost all top universities and research institutes have quickly established relevant laboratories and research centers. At the same time, many enterprises represented by the GelSight Company in the USA also quickly followed up and developed a series of products (Juyang et al., 1992). In the military field, the US Defense Advanced Research Projects Agency (DARPA) has set up several computational imaging related projects since 2007, such as “ARGUS-IS (Leninger et al., 2008),” “SCENICC (Sprague et al., 2012),” and “AWARE (Bar-Noy et al., 2011).” The North Atlantic Treaty Organization Science and Technology Agency also established a computational imaging task force in 2016, with defense units such as the US Army, Navy, Lockheed Martin, and the UK Ministry of Defense as the main members, and launched several projects such as SET-232 (Bosq et al., 2018).

In China, research on computational optical imaging is consistent with international activities. Corresponding laboratories and research centers were set up by Xidian University, Tsinghua University, Beijing Institute of Technology, and other universities, China Aerospace Science and Technology Corporation, Institute of Aerospace Information Research Institute, Xi'an Institute of Optics and Precision Mechanics, Changchun Institute of Optics, Fine Mechanics and Physics, and Chinese Academy of Sciences. To conduct research on computational optical imaging, the Computational Optical Imaging Technology Laboratory of the Institute of Aerospace Information Innovation, Chinese Academy of Sciences, has conducted extensive research in the fields of computational spectrum, light field, and active three dimensional (3D) imaging. The computational optical remote-sensor team at the Aerospace Information Research Institute developed and launched the world's first spaceborne computational spectral imaging payload (Liu et al., 2020). The National Information Laboratory and Institute of Optoelectronics Engineering at Tsinghua University have made important contributions (Cao et al., 2020). The Institute of Computational Imaging of Xidian University relies on the Key Laboratory of Computational Imaging of Xi'an to conduct research based on technologies such as scattered light imaging, polarization imaging, and wide area high-resolution computational imaging, and has obtained internationally recognized research results (Fei et al., 2019). The Optical Imaging and Computing Laboratory and the Measurement and Imaging Laboratory of the Beijing Institute of Technology have also proposed optimized solutions (Xinquan, 2007) for computational display and computational spectral imaging. The Intelligent Computational Imaging Laboratory of Nanjing University of Science and Technology has achieved excellent results in quantitative phase imaging, digital holographic imaging, and computational 3D imaging (Zhang et al., 2018). There are some notable research organizations in Europe that have been actively involved in computational imaging: Imperial College Computational Imaging Group (Imperial College London), Computational Imaging Group (University College London), Image and Video Analysis Group (Trinity College Dublin), Computer Vision Laboratory (ETH Zurich), Computer Graphics and Visualization Group (University of Zaragoza), Centre for Vision, Speech, and Signal Processing (University of Surrey), Max Planck Institute for Intelligent Systems (Germany), and Computer Vision and Image Processing Group (University of Verona).

Although CIT research continues, and many new imaging technologies have been derived, fragmented research has led to the difficulty of global systematic consideration of CIT, weak theoretical foundational support, and unclear application requirements. At the same time, the rapid development of Graphics Processing Unit (GPU) technology and advancements in Artificial Intelligence (AI) (Sinha et al., 2017; Barbastathis et al., 2019) have significantly contributed to the progress and application of computational imaging. In addition, as a research field covering many individual technologies, the current development ideas of CIT are disorganized. The complexity and breadth of the system make it difficult to present a clear research context, and the common basic problems and key technologies lack in-depth thinking. CIT is a type of target-oriented research technology, and its related research serves to develop or improve specific performance indicators and improve the target imaging quality by sacrificing other non-essential dimensions.

In summary, based on the demand orientation of CIT, the five application perspectives of “higher,” “farther,” “smaller,” “wider,” and “stronger” in future development are analyzed in this study. The characteristics, application prospects, and mutual relations of CIT are clearly defined, and new theories and ideas of CIT are discussed from another perspective, to promote the development of CIT in an orderly, systematic, and continuous manner.

The finiteness of the aperture of a photoelectric imaging system limits the imaging resolution. Traditional large-aperture photoelectric imaging systems under industrial design suffer from long development cycles, difficult processing and installation, high development costs, and poor environmental adaptability. In addition, they are prone to deterioration of the imaging quality owing to a decrease in the surface accuracy of the optical system. To solve this problem, it is necessary to develop a new type of ultra-large aperture optical imaging system, which is mainly realized using two synthetic aperture imaging technologies: primary mirror splicing and array complementary imaging. Compared with the traditional single-aperture photoelectric imaging system, the imaging resolution is higher, the mirror processing difficulty is lower, and the system is lighter.

In a primary mirror splicing space-based telescope, the primary mirror of the single-aperture telescope is divided into small pieces, with the pieces of sub-mirror spliced into an equivalent primary mirror, and folded in the fairing of the launch vehicle. After launch, the pieces are unfolded and reassembled through precision confocal adjustment and the resolution of the equivalent large-aperture telescope is achieved. As shown in Figure 1, the James Webb Space Telescope, which is the successor to the Hubble Telescope, was designed to use 18 hexagonal sub-mirrors with a diagonal distance of 1.5 m (Sabelhaus, 2004). However, even if primary mirror splicing technology is adopted, it is difficult to support a space telescope larger than 8–10 m in the short term until the splicing and folding technology has been improved.

Array complementary image telescope technology is also known as distributed synthetic aperture interferometry imaging technology (Changsheng et al., 2017). It collects light based on multiple small-aperture telescopes and then performs complementary imaging on the optical imaging telescope through an optical path delay line, baseline matching, beam pointing, and other relay optical systems. An imaging baseline is formed to achieve equivalent large-aperture effects. The larger the baseline, the higher the image resolution. These distributed devices can be launched separately, and assembled into a common frame structure in space. In theory, the array complementary image method can realize a larger aperture than the primary mirror splicing technique.

In Xiang et al. (2021) proposed a coherent synthetic aperture imaging system. Figure 2 shows a schematic diagram of the system in a scene of two synchronous orbit satellites, one of which is equipped with a camera, and the other with a laser source for angle-changing illumination.

To achieve imaging beyond the diffraction limit, novel imaging methods, such as structure illumination microscopy (Gustafsson, 2000), stochastic optical reconstruction microscopy (Rust et al., 2006), negative refractive super lenses (Pendry, 2000), and ptychography imaging (Wang et al., 2022), have been proposed. The ptychography imaging concept was first proposed by HOPPE W in 1970s (Hegerl and Hoppe, 1972). The core of this method is to search for a unique complex solution to satisfy the constraints of multiple far-field diffracted intensity images in overlapping scanning modes. An iterative algorithm of phase restoration is used, which sacrifices the time dimension and thus achieves an imaging result with an ultra-diffractive resolution limit. In Zheng et al. (2013) proposed the Fourier ptychographic microscopy technique that uses an objective lens with lower magnification to simulate the performance of an objective lens with higher calculated magnification. Subsequently, a reconstruction algorithm was adopted to recover the complex amplitude information of the object and obtain high-resolution images. A conventional Fourier ptychographic imaging system uses a light emitting diode (LED) plate as an illumination source. Figure 3 shows a schematic of a conventional Fourier ptychographic imaging system (Jiasong et al., 2016) and the reconstructed images, which have a much higher imaging resolution than conventional optical microscopic imaging. Although imaging beyond the diffraction limit has been accomplished in the field of microscopic imaging, there has been no similar breakthrough in large-scale macroscopic imaging applications. This is also an urgent problem that needs to be solved using computational imaging in pursuit of higher imaging resolution.

In addition to the non-interference computational imaging microscopy techniques introduced above, interferometric computational imaging techniques (Park et al., 2018) such as synthetic aperture quantitative phase imaging (Cotte et al., 2013) have doubled the maximum spatial frequency as shown in Figure 4. Alexandrov et al. (2006) introduced a novel synthetic aperture optical microscopy technique that generates high-resolution, wide-field images in both amplitude and phase using Fourier holograms. The spatial and spectral qualities of the illumination field, as well as the collection and solid angles, determine the part of the complex two-dimensional spatial frequency spectrum of an object that is captured by each hologram. They showcased the use of synthetic microscopic imaging to capture spatial frequencies that are beyond the modulation transfer function of the collection optical system, all while maintaining a long working distance and wide field of view. While its capabilities are restricted by the numerical aperture of the objective lens.

Super-resolution reconstruction is a type of information processing technique that uses low-resolution image recovery to obtain high-resolution images, and was first proposed by Harris (1964). It depends on the number of raw low-resolution images that can be classified as single-image and multi-image super-resolution. In Dong et al. (2014) applied deep learning to a natural image super-resolution reconstruction procedure and proposed a super-resolution convolutional neural network (SRCNN) model, as shown in Figure 5. To address the problem of the weak learning ability of the SRCNN shallow model, Kim et al. (2016) proposed a very deep super-resolution network model that included 20 convolutional layers. The use of a deeper network model improved the reconstruction effect. However, a deeper network model results in slower convergence speed. Lim et al. (2017) proposed an enhanced deep super-resolution reconstruction network model with 69 convolutional layers. This method reduced the memory requirements by approximately 40% and improved the convergence speed by improving the residual. The restoration results are shown in Figure 6.

However, because the super-resolution reconstruction technology relies only on subsequent data processing, the result of the super-resolution reconstruction is different from the real value. The next research direction is to organically combine super-resolution reconstruction with the imaging process to achieve real-time super-resolution.

An optical imaging system that is exposed to bad weather conditions such as fog, haze, rain, or snow, or even to underwater imaging conditions, cannot obtain the target information directly owing to the random transmission of photons, which can only result in an irregular distribution of the scattered light field. Imaging through scattering technology is a mainstream imaging technology that can recover clear target information through the deep interpretation of scattered light images carrying hidden target information. Currently, imaging through scattering technology includes wavefront shaping (Katz et al., 2011), optical memory effect (OME)-based nonvisual imaging (Osnabrugge et al., 2017), and deep learning methods.

Polarization plays an irreplaceable role in the study of descattering. The current research shows that the polarization distribution characteristics of a scattered light field are closely related to the imaging distance. The polarization statistical characteristics of a scattering medium, such as water, were studied, and the intensity and polarization characteristic distribution of the scattered light field were considered globally to address the long-distance imaging problem affected by the medium.

Insect compound eyes are small, have a large field-of-view angle, and are sensitive to high-speed moving objects. Inspired by their unique imaging modes, the compound eye-like optical system achieves wide-area high-resolution imaging by mimicking biological visual mechanisms, which significantly improves the imaging performance of optoelectronic imaging and detection equipment. In Brady and Hagen (2009) proposed the TOMBO compound eye imaging system, which was designed using the juxtaposed compound eye structure of dragonflies as a reference, as shown in Figure 18. Each aperture images the full field of view. Compared with traditional single-aperture imaging, the multi-aperture imaging method effectively improves the information capacity of the system. Although multi-scale imaging is obviously different from the multi-aperture imaging approach, the multi-scale system adopts a multi-stage system cascade to achieve high-performance imaging, and most of the more mature imaging systems currently contain only two stages (Figure 18C), large-scale primary optics and small-scale secondary optics. The large-scale primary optics are used mainly to collect as much light energy as possible and to perform the initial aberration correction, whereas small-scale secondary optics are used mainly to secondarily transmit the light passing through the primary optics and image it onto the detector behind the secondary optics. Overall, however, the compound eye-like optical imaging technology not only effectively solves the problem of constraints between a large field of view and high resolution but also significantly reduces the size, weight, and power consumption of the imaging system.

Since 2012, the Swiss Federal Institute of Technology in Lausanne has successively developed Panoptic (Afshari et al., 2012; Popovic et al., 2014), OMNI-R (Akin et al., 2013), GigaEye-1 (Cogal et al., 2014), GigaEye-2 (Popovic, 2016), and other multi-aperture imaging systems, among which, the OMNI-R has a full-field-of-view angle as high as 360° × 100°, and its structure is similar to that of GigaEye-1. However, GigaEye-1 supports two imaging modes and exhibits good imaging effects for both static and dynamic scenes. In 2017, an ultra-compact high-definition imitation compound eye system was developed (Cogal and Leblebici, 2016), which has a pixel count of up to 1.1 million pixels while achieving full-field-of-view of 180° × 180° imaging, and the radius of the whole system is only 5 mm. The system is equipped with a distributed illumination system, which is able to achieve dark-environmental imaging. In 2014, a new astronomical telescope was designed by Law et al. (2014, 2015) that can image an area of 384 square degrees and detect up to 16 magnitudes, which greatly improves the telescope's imaging range and detection capability.

In a traditional optical system design, the imaging link has a one-way design and independent optimization. This means that the optical design and image-processing algorithms are independent of each other, and the imaging link cannot be considered as a whole. Hence, it is easy to miss the optimal scheme for the joint design of an optical system and image processing. The minimalist optical system comprehensively considers the entire imaging link and achieves the purpose of simplifying the optical system structure and reducing costs based on the idea of global optimization to better promote the engineering application of the optical system. A comparison of the two design processes is shown in Figure 19A (Stork and Robinson, 2008).

In Robinson and Stork (2006) proposed a method for the joint design of an optical system, detector, and image processing for document scanners and other instruments. The method uses the mean square error (MSE) of the predicted recovered image and the optically blurred image as an evaluation index and combines it with the Wiener filtering-based image recovery algorithm to recover optically blurred images. The effect of the recovered image was better than that of the traditional design (Robinson and Stork, 2007, 2008), as shown in Figure 19B. In 2008, Robinson and Stork proposed the idea of co-designing an optical design with image restoration, which achieved end-to-end optimization by combining an optical transfer function with an image processing system using the MSE between the restored and original target images. In Li J. Y. et al. (2021) combined Zemax software and image processing through data exchange dynamic link communication technology to form a closed-loop link for optical-algorithmic joint design optimization, which reduced the difficulty of the optical system design with the system volume, weight, and cost, as shown in Figure 19C. The joint design of a simple lens can be comparable to the imaging quality of a traditionally designed complex lens.

Computational optical system design technology effectively solves the shortcomings of the traditional optical system with complex internal structure, large volume, and high cost through the whole link integration optimization design and provides strong technical support for the miniaturization, light weight, and portability of photoelectric imaging equipment.

Traditional photodetectors are based on photosensitive semiconductors. The thermal sensitivity, negative resistivity, temperature characteristics of semiconductors, and quantum effects generated for detectors of small sizes affect the detection efficiency, thus affecting the photoelectric imaging ability. Therefore, it is necessary to find new photosensitive materials and create new photosensitive components to overcome the material limitations of silicon-based semiconductors to improve the detection sensitivity and reduce the detection threshold. At present, photodetectors can only respond to light intensity. With the attenuation of light waves through long-distance transmission, the light intensity information reaching the detector is limited, and more is needed to retain the physical quantity information of other dimensions. The development of a detector with multi-dimensional physical quantity response is of great significance for achieving remote imaging.

A traditional photodetector has a fixed plane structure and depends on the correction compensation of the optical system to obtain the ability of the focusing plane target surface. This can be achieved by using different lens combinations, resulting in a complex system, image distortion, and quality degradation problems. The retinal structure of the human eye is concave, allowing imaging to be realized by relying only on the relatively simple optical structure of the lens. Inspired by this, if an imaging detector with a flexible curved surface is used, as shown in Figure 20, and the adaptive layout is conducted according to the focal plane shape of the optical system, the correction pressure of the optical system's imaging distortion can be reduced, the optical system design and complexity can be simplified, and high-resolution performance can be achieved (Zhang et al., 2017). However, current processing technology limits the preparation of curved surface detectors, and improving the process is major difficulty.

In the process of converting incident photons into discrete digital signals that can be processed by a computer, it is necessary to perform signal sampling; Nyquist sampling used by traditional detectors causes considerable data redundancy and time-consuming data processing. Therefore, it is necessary to design a new non-uniform sampling computing detector and establish a matching computational optical system design scheme so that the significance of detection and sampling focuses on the target information to improve the imaging resolution and signal processing speed. In addition, it is necessary to develop an optical processing method that can transfer the information calculated from the electrical signal processing at the back end of the detector to the front end of the detector so that the detector itself can preprocess the imaging information.

Single-scale multi-aperture imaging is a technique used to improve and realize the function of an imaging system by mimicking biological visual mechanisms, such as the widely used fisheye lens. To achieve a large field of view and high-resolution imaging, a biomimetic multi-aperture imaging system was developed by drawing on the structure of the compound eye of arthropods, which was first proposed by Brady and Hagen (2009), to some extent solving the problem of the incompatibility of a large field of view and high resolution.

In Akin et al. (2013) developed a high-resolution imaging system inspired by the panoramic optics approach, capable of omnidirectional video recording at 30 frames per second and a resolution of 9,000 × 2,400. The imaging field of view of the system reached 360° × 100°, and the physical and photographic samples of the system are shown in Figure 21A.

ARGUS-IS (Leninger et al., 2008), an aerial camera system jointly developed by DARPA in the USA and Aerospace Systems in the UK in 2013, has a strong imaging reconnaissance capability. With 1.8 billion pixels, the system can monitor an area of more than 24 square kilometers from an altitude of 5.4 km, and detect more than 40,000 moving objects, including the identification and calibration of moving people and cars. It also has powerful information storage capabilities, as shown in Figure 21B. Its imaging resolution is sufficient to identify and track vehicles and pedestrians from an altitude of 6,500 m, ground resolution of 0.15 m, instantaneous field of view angle of 23 μrad, and can simultaneously track at least 65 targets.

In Fu et al. (2015) designed the first generation of a bionic compound eye optical system with a large field of view and low edge aperture resolution. However, the central aperture had the opposite effect (Kitamura et al., 2004). The 31 components detected the target using the edge aperture and accurately identified the target using the center aperture with a full field of view of 53.9°. In Shao et al. (2020) research group designed a multi-aperture system with a full field of view of 123.5° × 38.5°. The system also supported real-time viewing and other functions, such as images, videos, and other information. The prototype and imaging results are shown in Figure 21C.

Since Brady and Hagen (2009) proposed the theory of multi-scale imaging in 2009, this imaging method has received extensive attention from researchers worldwide. Since 2012, the AWARE-2 (Golish et al., 2012; Youn et al., 2014), AWARE-10 (Nakamura et al., 2013; Marks et al., 2014), and AWARE-40 (Nakamura et al., 2013) were developed. It takes only 18 s to actually shoot a single photograph, which effectively realizes high-resolution imaging with a large field of view. As an improved version of AWARE-2, AWARE-10 has a field of view of 100° × 60°, two billion pixels, and a resolution of 12.5 cm @ 5 km, representing a significant improvement in the number of pixels and resolution compared to AWARE-2.

In Shao et al. (2020)'s team developed a prototype multi-scale wide-area high-resolution computational optical imaging system based on the design principle of the secondary imaging system, as shown in Figure 22A (Fei et al., 2019). It had an imaging field of view of 120° × 90°, a system pixel count of 3.2 billion, and a resolution of 5 cm @ 5 km, which was capable of clearly resolving target objects within a range of 5 km. It was suitable for applications such as key area defense, border patrol, long-distance detection, and social activity surveillance. In the future, such systems can also play an important role in airborne, ground-based, and super-converged real-view reconnaissance. In the same year, Shao Xiaopeng's team designed a multi-scale system in the infrared band with a range of 8–12 μm. The system had a magnification ratio of 2 × and a focal length of 68–136 mm. The system resolution was 0.179 mrad in the telephoto mode, and 0.36 mrad in the short focal length, which was capable of effectively realizing the acquisition of targets in a large field of view and the identification of targets in a small field of view with high precision. The following year, the team miniaturized the multi-scale computational optics system, taking advantage of Galileo's compact structure to reduce the volume and complexity of the computational optics system. This resulted in a significant reduction in cost and energy consumption, and greater applicability when the system was engineered for application.

On the basis of AWARE-2 (Golish et al., 2012; Youn et al., 2014) and AWARE-10 (Nakamura et al., 2013), Brady developed an AWARE-40 (Nakamura et al., 2013; Marks et al., 2014) imaging system. Unlike AWARE-2 and AWARE-10, the primary mirror in AWARE-40 adopted a double Gaussian-like structure instead of a spherical lens, and its pixel count was up to 3.6 billion, with a resolution of up to 5.4 cm @ 5 km. The system had a superior imaging performance and could clearly detect and identify long-distance targets. The prototype and the imaging results are shown in Figure 22B. In Wu et al. (2016) comprehensively considered light, light field information, sensors, and image reconstruction in the computational imaging process and used 4D deconvolution algorithms based on a multi-scale imaging design scheme to obtain a large-field-of-view high-definition image suitable for biomedical applications.

Hyperspectral imaging uses narrow and continuous spectra for the continuous remote sensing imaging of targets, which has great advantages in the fine classification, matching identification, and analysis of distant targets. However, the global scanning of multiple spectral bands results in tens or even hundreds of gigabytes of data per spectral data cube. Hence, rapidly locating the target information of interest from a large amount of data is a problem that needs to be solved by hyperspectral imaging technology in the future. In addition, the traditional scanning imaging method of hyperspectral imaging results in poor real-time imaging.

In Kumar et al. (2017) proposed a single-exposure multispectral imaging technique using a single camera. It sacrificed spatial dimensional information using spatial correlation and spectral decorrelation of scattered images to achieve multispectral imaging of simple structures, as shown in Figure 23.

In Monakhova et al. (2020) presented a system comprising an array of tiled spectral filters placed directly on an image sensor and a diffusion plate placed close to the sensor. Each point on the diffusion plate plane was mapped to a unique pseudorandom pattern on the spectrum filter array, which encoded multiplexed spatial spectral information. By solving the sparse constraint inverse problem, hyperspectral images were recovered at a sub-super-pixel resolution.

A spectral detector based on a monochrome camera can achieve spectral imaging with a compact and simple structure, and its cost is significantly less than that of a traditional spectral imaging system. However, the trade-off between spectral resolution and spectral range and the energy transmittance problem still exists, and the related theoretical supplement is the key research direction of spectral imaging technology in the next step.

The existing 3D imaging methods are limited by the means of interpretation and imaging equipment, and cannot satisfy the increasing demand for 3D imaging in many different application scenarios. It is difficult to achieve the universal high-precision imaging of long-range targets. Polarized 3D imaging technology, through the study of the object surface morphology and polarization characteristics of the reflected light, interprets the multi-physical information of the optical field and achieves high-precision reconstruction of the target (Miyazaki et al., 2016). The separation of the specular-diffuse reflection and the multi-value of the azimuth angle in the polarized 3D imaging method have been the core problems restricting its development in Figure 24A. In Miyazaki et al. (2002a) proposed a rotational measurement method to eliminate the multi-value of the incidence angle in specular reflection and then proposed visible and infrared wavelength measurement methods (Miyazaki et al., 2002b), which effectively avoided the multi-value of the incidence angle. With the help of the polarization of far-infrared wavelengths and high-precision measurement of the angle of incidence, they solved the multi-value problem of the angle of incidence. However, methods using far-infrared and visible light band measurement are complex and expensive. Moreover, different bands must be resolved under image matching and other issues, making the process cumbersome and complex. Many studies have proposed effective solutions to the multi-value problem of the incidence angle. On the basis of accurately obtaining the target angle of incidence, eliminating the multi-value problem of azimuth has become another challenge for researchers. Morel et al. (2006) proposed an active illumination method for the multi-value azimuth. This method solves the multi-value azimuth problem by modulating the light source in different directions; however, it cannot measure the azimuth of a moving target. Zhou et al. (2013) proposed a method to diffuse the azimuth information from the high-frequency region to the low-frequency region to solve the multi-value problem in the low-frequency region, which can achieve high-precision 3D reconstruction of the target on complex surfaces.

In Kadambi et al. (2017) acquired a surface depth map of a target object using Kinect and then fused it with the polarized 3D imaging technique. They effectively recovered the detailed information of the object's surface depth map while solving the problem of a non-unique azimuthal angle and realized high-precision 3D imaging under different target conditions. The reconstruction results are shown in Figure 24B. However, this method is limited by the distance of the Kinect device and it is difficult to achieve high-precision imaging over long distances. Subsequently, Han et al. (2022) combined a deep learning approach with a monocular polarization camera to simultaneously achieve specular-diffuse reflection separation and azimuthal correction to achieve millimeter-level 3D imaging accuracy at a long distance, as shown in Figure 24C. In addition, Li X. et al. (2021) proposed a near-infrared monocular 3D computational polarization imaging method to improve the material universality. They introduced a reference gradient field in the weight constraints to globally correct the surface normal blurring of the target with inhomogeneous reflectivity, which realized the direct shape reconstruction of inhomogeneous surfaces with inhomogeneous reflectivity. This method is simple, robust, and effectively avoids the influence of changes in the reflectivity.

By increasing the dimensions of the light field information, polarization 3D imaging technology solves the problem of mutual restriction between the imaging distance and 3D accuracy in 3D imaging and effectively improves the universality of 3D imaging technology.

This improvement in the physical dimensions can relax, and achieve a universal expansion of, the imaging conditions. For extremely low-SNR decoding technology under extreme imaging conditions, an increase in the mathematical dimension is also a major development idea. A typical example is the sparse low-rank decomposition of signals, and its core introduces a sparse low-rank matrix decomposition model in mathematics for signal detection. The slow-varying background term is regarded as a low-rank term, and the weak man-made target term is regarded as a sparse term. Accordingly, with the help of the optimization model the SNR of the target signal can be greatly enhanced. Then, through the separation of the information and inverse transformation, the detection of the weak target and optical field disturbance signal in a specific domain can be achieved, which greatly improves the imaging universality. This principle is illustrated in Figure 25A (Zutao et al., 2016).

Because the solution of the sparse low-rank problem is pathological, in Cao et al. (2017) proposed a low-rank sparse decomposition model based on the truncation norm to detect the background and foreground of datasets from different video databases. The recovered background and foreground contain no noise, which meets practical requirements. Part of the processing effect is shown in Figure 25B. This idea was then used to process the face image to effectively remove shadows and reflections from the image, and the processing effect is shown in Figure 25C.

Subsequently, Fei et al. (2021) applied sparse-low-rank decomposition to underwater imaging for the first time and proposed an underwater polarimetric imaging technique based on sparse-low-rank characteristics. They established a sparse low-rank decomposition model for underwater images in the polarimetric domain, effectively separated the target and background information, and reconstructed a high-definition target image to achieve high-quality recovery of the image under the conditions of a low SNR. The imaging results are shown in Figure 25D. In the same year, Daubechies et al. (2004) and Berman et al. (2016) proposed a low-rank and dictionary expression decomposition method for haze removal in dense fog scenes, by constructing a low-rank and dictionary expression decomposition model to obtain a low-rank “haze” map. They then recovered a clear image using double and triple interpolation, producing a haze removal effect in different scenes, as shown in Figure 25E. This proves that the proposed method yields satisfactory results for hazy images in different scenes.